Provisioning of a shippable storage device and ingesting data from the shippable storage device

ABSTRACT

When a client requests a data import job, a remote storage service provider provisions a shippable storage device that will be used to transfer client data from the client to the service provider for import. The service provider generates security information for the data import job, provisions the shippable storage device with the security information, and sends the shippable storage device to the client. The service provider also sends client-keys to the client, separate from the shippable storage device (e.g., via a network). The client receives the device, encrypts the client data and keys, transfers the encrypted data and keys onto the device, and ships it back to the service provider. The remote storage service provider authenticates the storage device, decrypts client-generated keys using the client-keys stored at the storage service provider, decrypts the data using the decrypted client-side generated keys, and imports the decrypted data.

This application is a divisional of U.S. patent application Ser. No.14/975,363, filed Dec. 18, 2015, now U.S. Pat. No. 9,934,389, which ishereby incorporated by reference herein in its entirety.

BACKGROUND

Growth of data storage capacity for computer systems has far outpacedthe growth in transmission speed for transferring data over networksbetween computer systems. The discrepancy is so great that transmittinga large amount of data from one storage facility to another storagefacility can be prohibitively costly (e.g., requiring costly systemupgrades) or lengthy (e.g., transmission taking several months orlonger). Physically moving the storage media may leave the data onlegacy hardware or may not be an available option (e.g., when the datais stored by a storage service on behalf of the customer). Somesolutions have involved transferring the data to a portable storagedevice (e.g., network attached storage devices) and shipping theportable storage device to another storage facility where the data istransferred to another storage system.

For example, when a customer of a storage service provider wishes tomove a large quantity of data from the customer's site to a location atthe storage service provider, the customer may save the data onto adevice and ship the device to the storage service provider. However, theconfidentiality of the data may be compromised for various reasons. Forexample, during shipment, mistakes may occur that prevent a storagedevice from being shipped to the correct destination. Moreover, thedevice may be intercepted by a third party. Thus, a malicious thirdparty may access confidential data on the device. In some cases, thedevice may arrive at the storage service provider without any indicationthat unauthorized access occurred.

Further, different customers may use different types of storage devicesto transfer data to the storage service provider. New storage devicesand techniques are constantly being developed and adopted by customers.Therefore, as the amount of data transferred from customers grows, itmay become increasingly difficult for a storage service provider totransfer the data from multiple disparate storage devices in a secureand efficient manner.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a system and a process for securely transferring datausing a shippable storage device, according to some embodiments.

FIG. 2 illustrates a shippable storage device, according to someembodiments.

FIG. 3 is a logical block diagram of a shippable storage device,according to some embodiments.

FIG. 4 is a flow diagram of a process of securely transferring datausing a shippable storage device, according to some embodiments.

FIG. 5 is a logical block diagram illustrating a secure data transferscheme using a shippable storage device, according to some embodiments.

FIG. 6 is a flow diagram of a process for collecting information for anew data import job to securely transfer data using a shippable storagedevice, according to some embodiments.

FIG. 7 is a flow diagram of a process for provisioning a shippablestorage device to securely transfer data for a data import job,according to some embodiments.

FIG. 8 is a flow diagram of a process for sending a shippable storagedevice and a manifest to a customer, according to some embodiments.

FIG. 9 is a logical block diagram of a shippable storage device readyfor shipping to a customer, according to some embodiments.

FIG. 10 is a flow diagram of a process for receiving a shippable storagedevice and information for a data import job, according to someembodiments.

FIG. 11 is a logical block diagram of shippable storage device connectedto a customer network, according to some embodiments.

FIG. 12 is a flow diagram of a process for decrypting a manifest andauthenticating a shippable storage device, according to someembodiments.

FIG. 13 is a logical block diagram of a manifest including informationassociated with a data import job, according to some embodiments.

FIG. 14 is a flow diagram of a process for storing encrypted data onto ashippable storage device, according to some embodiments.

FIG. 15 is a flow diagram of a process for encrypting data and keys andstoring the encrypted data and encrypted keys onto a shippable storagedevice, according to some embodiments.

FIG. 16 is a flow diagram of a process for storing encrypted shards ontodifferent shippable storage devices, according to some embodiments.

FIG. 17 illustrates a process for encrypting chunks using chunk keys,according to some embodiments.

FIG. 18 illustrates a process for encrypting chunk keys using a filekey, according to some embodiments.

FIG. 19 illustrates a process for encrypting file keys using a bucketkey, according to some embodiments.

FIG. 20 is a logical block diagram of relationship between encryptionkeys associated with a data import job, according to some embodiments.

FIG. 21 is a logical block diagram of a shippable storage device readyfor shipping to a storage service provider, according to someembodiments.

FIG. 22 is a flow diagram of a process for receiving a shippable storagedevice at a storage service provider and ingesting secure data from theshippable storage device, according to some embodiments.

FIG. 23 is a flow diagram of a process for decrypting and importing datafrom a shippable storage device at a storage service provider, accordingto some embodiments.

FIG. 24 is a flow diagram of a process for wiping a shippable storagedevice at a storage service provider, according to some embodiments.

FIG. 25 is a block diagram illustrating an example computer system thatimplements some or all of the techniques described herein, according tosome embodiments.

While embodiments are described herein by way of example for severalembodiments and illustrative drawings, those skilled in the art willrecognize that embodiments are not limited to the embodiments ordrawings described. It should be understood, that the drawings anddetailed description thereto are not intended to limit embodiments tothe particular form disclosed, but on the contrary, the intention is tocover all modifications, equivalents and alternatives falling within thespirit and scope as defined by the appended claims. The headings usedherein are for organizational purposes only and are not meant to be usedto limit the scope of the description or the claims. As used throughoutthis application, the word “may” is used in a permissive sense (i.e.,meaning having the potential to), rather than the mandatory sense (i.e.,meaning must). Similarly, the words “include,” “including,” and“includes” mean including, but not limited to.

DETAILED DESCRIPTION OF EMBODIMENTS

The systems and methods described herein implement secure data transferfrom one location to another location using a shippable storage device.Due to limited transmission speeds over networks, a large amount of datamay be transferred from one location to another in a much shorter amountof time by using a shippable storage device instead of by using networktransmission. Techniques for securing data stored on the shippablestorage device may be implemented to ensure that the data will not beexposed to a third party if the shippable storage device is misplaced orintercepted during shipment from one location to another location.

FIG. 1 illustrates a system and a process for securely transferring datausing a shippable storage device 100, according to some embodiments. Theillustrated embodiment depicts movement of the shippable storage device100 from a location of a storage service provider 102 to a location of acustomer network 104 and back to the storage service provider 102 aswell as various processes A-G that are performed along the way byvarious entities. The shippable storage device 100 depicted in FIG. 1may be the same as the shippable storage device 100 depicted in FIGS. 2,3, 5, 9, 11, and 22, in embodiments.

In some embodiments, a customer may have other location with networksand the shippable storage device 100 may be shipped to those locationsinstead of, in addition to, before, or after the depicted shipment C. Insome embodiments, multiple different shippable storage devices may beshipped to various locations at the same time as the depicted shipment.Shippable storage devices 100 may be shipped to entities with networksother than customers, for example but not limited to, various otherenterprises, government entities, other service providers or the like.Computers, such as a server or desktop computers at the location of thecustomer network 104 may perform some or all of the processesillustrated in FIGS. 4, 10, 12, 14-19, 21, and 18, in embodiments.Computers, such as a server or desktop computers at the location of thestorage service provider 102 may perform some or all of the processesillustrated in FIGS. 4, 6-8, and 23-25, in embodiments.

In the depicted embodiment, (A) a customer creates, generates orrequests that a data import job be performed. The data import jobcreation or request may be performed via a console interface such as anetwork-based page or site provided by the storage service provider 102that the customer accesses via a computing device over one or morenetworks (e.g., network 106). At (B) the storage service provider 102provisions a particular shippable storage device 100, instructs theshippable storage device 100 to display an address associated with thecustomer (e.g., obtained during job creation (A)) and ships theshippable storage device 100 to the requesting customer network 104according to data import job information (e.g., job information may bedetermined during generation of the job at request time, in someembodiments).

At (C), a display 108 of the shippable storage device 100 is updated todisplay the address of the customer network 104 and the shippablestorage device 100 is shipped to the location of the customer network104. In some embodiments, the shippable storage device 100 does not havea display 108 or does not use the display 108. In such cases, an addressmay be written, stamped, or otherwise affixed onto the shippable storagedevice. In some embodiments, the shippable storage device 100 is shippedwithin an enclosure, which has the address visible on an outer portionof the enclosure.

At (D), the customer network 104 downloads a data transfer tool via thenetwork 106 that is configured to encrypt customer data and store theencrypted data onto the shippable storage device 100. The customernetwork 104 also downloads a manifest for the data import job via thenetwork 106, which may include encryption keys and other metadataassociated with the data import job. In some embodiments, the datatransfer tool downloads the manifest. At (E) the data transfer toolencrypts customer data and stores the encrypted customer data onto theshippable storage device 100. At (F), the display 108 of the shippablestorage device 100 is updated with the address of the storage serviceprovider 102 and the device is shipped back to the storage serviceprovider 102. For example, the storage service provider 102 may send anupdated address via the manifest, or the device may recognize it is atthe customer location and automatically update the displayed destinationaddress from a memory store programmed with the next destination duringthe provisioning step (B).

In some embodiments, the shippable storage device 100 may be sent to oneor more other customer sites to have additional data stored on theshippable storage device 100 before being sent back to the storageservice provider 102. At (G) the shippable storage device 100 isreceived back at the storage service provider 102 and connected to anetwork of the storage service provider 102. The storage serviceprovider 102 may then verify that the shippable storage device 100 hasnot been tampered with. For example, the storage service provider 102may verify that a configuration of the shippable storage device 100 hasnot been changed since the customer data was stored on the shippablestorage device 100. The customer data is then ingested from theshippable storage device 100 into the storage service provider 102system. The shippable storage device 100 is then wiped and stored untilit is ready to be provisioned for another job.

In some embodiments, the shippable storage device 100 may be sent ontoother customer locations or other customers distinct from the customerto store additional data before the shippable storage device 100 is sentback to the storage service provider 102. In some instances, theshippable storage device 100 may be used to distribute data from thestorage service provider 102 to one or more different sites of the samecustomer, in embodiments. The shippable storage device 100 may beinstructed to display various addresses at various times, such that theshippable storage device 100 is used to multi-cast data, for example ineither the export or import embodiments. It is contemplated that thevarious addresses may be stored on the shippable storage device 100 atonce, such as when the shippable storage device 100 is provisioned, forexample, or the various addresses may be sent to the shippable storagedevice 100 over a network, such as a cellular network or via a customeror provider network or combination thereof. In embodiments, theshippable storage device 100 may include logic configured to update thedisplayed address based on the shippable storage device 100 sensing ageographical position or location of the shippable storage device 100.For instance, a shippable storage device 100 that determines (e.g.,based on a GPS sensor or cellular triangulation) that it has arrived atone location, may be configured to display the next address stored inmemory of the device.

Please note that previous descriptions are not intended to be limiting,but are merely provided as examples of securely transferring data usinga shippable storage device. For instance, one or more steps may beremoved and/or one or more steps may be added to securely transfer datausing a shippable storage device. Moreover, in various embodiments,steps may be performed in different sequences.

This specification next includes a description of a shippable storagedevice that may be used for securely transferring data from one locationto another, such as from a customer to a storage service provider (orvice versa). Then, an overview of the process of securely transferringdata using a shippable storage device is provided. Next, a diagramillustrating a secure data transfer scheme is provided, includingdifferent components that may be employed as part of implementing thesecure data transfer scheme. A number of different techniques to performsecure data transfer are then discussed, some of which are illustratedin accompanying diagrams and flowcharts. Finally, a description of anexample computing system upon which the various components, modules,systems, devices, and/or nodes may be implemented is provided. Variousexamples are provided throughout the specification.

FIG. 2 illustrates a shippable storage device 100, according to someembodiments. The depicted shippable storage device 100 may be used tomove large amounts of customer data off of customer storage networks orservers to other storage networks or servers, when other forms oftransfer (e.g., broadband data transmission) are unavailable or cost ortime prohibitive, for example. Embodiments of the shippable storagedevice 100 may include more, less, or different features or componentsthan those depicted, in embodiments.

In the depicted embodiment, shippable storage device 100 includes anenclosure 215 surrounding persistent storage 290. The persistent storagemay include any type of storage such as, but not limited to hard diskdrives, optical media, magnetic tapes, memristor storage, persistent RAMor solid state storage devices. The enclosure may be ruggedized (e.g.,according to various standards, such as military standards orelectronics industry standards) and may be configured with anoutward-facing electronic display 214 such that when enclosed by theenclosure, the persistent storage, the enclosure, and the electronicdisplay form a self-contained shipping container suitable for shippingwithout any additional packaging, labeling or the like and such that theelectronic display 214 acts as to display a destination location (e.g.,in lieu of a shipping label). In embodiments, the enclosure 215 and thedisplay 214 act as reusable shipping components in lieu of cardboardboxes and shipping labels. The enclosure may include various mechanismsto facilitate movement of the shippable storage device 100, such asrollers, handles or the like.

The shippable storage device 100 is illustrated with battery 260 andpower connection 250 for powering some or all of the components of theshippable storage device 100 that require power to function. The powerconnection 250 may be configured to connect the shippable storage device100 to an external power source, in embodiments. The power connector maypower the persistent storage, in some embodiments. Other sources ofpower are contemplated, such as kinetic energy sources that rely uponthe motion during shipping to power the shippable storage device 100,solar energy sources, or the like. Any of various power sources maypower the electronics (e.g., the display or the storage) of theshippable storage device 100.

The shippable storage device 100 is depicted with display 214. Thedisplay 214 may incorporate any of various display technologies, such aslow-power electronic-ink (E-ink), organic light emitting diodes (OLED),active-matrix organic light-emitting diode (AMOLED), flexible displaysor touch-sensitive displays as non-limiting examples. Low-power e-inkdisplays may provide the benefit of reduced power consumption for ashipping environment where small batteries (e.g., batteries that costless to ship, are less expensive or take up less shipping space) arepreferred. The shippable storage device 100 may be configured withmultiple displays 214, in some embodiments. For example, some carriersor fulfillment centers label three sides of a shipping container suchthat the destination of the container can be scanned or readirrespective of the orientation of the container. Similarly, multipledisplays can be incorporated into multiple sides of the enclosure 215 ofthe device. For example, the enclosure may be configured with 1-6 ormore displays, in some embodiments. The various displays maybeconfigured such that the displays are computer readable (e.g., viascanner).

The shippable storage device 100 is illustrated with network interface240. The network interface 240 may act as interface between theshippable storage device 100 and various networks, such as LANS, WANS orthe like (e.g., via various protocols, such as iSCSI or Ethernet). Insome embodiments, network connection 240 may act as an interfacedirectly to another device (e.g., via SCSI). In some instances, thenetwork interface 240 may include two or more different types ofinterfaces (e.g., RJ45, SFP, optical).

The shippable storage device 100 is illustrated with switch 230. Theswitch 230 may act as an on-off power switch or as a switch to activatethe display, in some embodiments. Device 100 is also illustrated withantenna 220. The antenna may be configured to facilitate wirelesscommunication between the service provider or customer and the device.For example, the wireless communication may be over various cellularnetworks, Wi-Fi, or the like (e.g., network 106). For instance, theservice provider may send updated address information to the shippablestorage device 100 via cellular networks while the shippable storagedevice 100 is en route to some location. The updated address informationmay be displayed via the display 214 such that the shippable storagedevice 100 is rerouted on the fly, for example. In other embodiments,the wireless communication channel may be used to send updated shippinginformation for display while the device is located at the customersite. In embodiments, cellular networks may be used to track the device.

The shippable storage device 100 is illustrated with radio frequencyidentification (RFID) 280. The RFID may assist with tracking the device,in some instances. For example, devices may be identified during theprovisioning process via a respective RFID or devices may be identifiedupon receipt at the customer or upon return to the service provider by arespective RFID. The RFID may be used to track the shippable storagedevice 100 as the device is routed through a facility, such as through aservice providers fulfillment facility (e.g., while routed on a conveyorsystem).

The shippable storage device 100 is illustrated with various sensors222, 224. The device may be outfitted with any of various sensorsincluding a global positioning sensor (GPS), a temperature sensor, ahumidity sensor or an accelerometer, all as non-limiting examples. Datamay be collected from the sensors and used in various manners, such asto record the environment of the device (e.g., hot, cold, moderate,moist) or record various events associated with the shippable storagedevice 100, such as a drop, quick movement, orientation or location ofthe shippable storage device 100. The sensor data may be stored locally,sent over the network 130 or displayed via display 214.

The shippable storage device 100 may be configured with multiple layersof security. For example, data stored on the device may be encrypted oneor more times, with one or more keys. The keys may be determined,stored, controlled or held by various parties and applied at varioussteps of the illustrated processes. For example, some keys used toencrypt the data stored on the device may be stored separate from thedevice, while other keys used to encrypt the data on the device may bestored with the device. The encryption keys may be applied in multiplelayers, in embodiments.

The shippable storage device 100 may be configured as one or more othertypes of network-based device or other electronic devices, such astransient local hardware for example. In an example, non-exhaustivelist, device 100 may be configured as various combinations ofcryptographic hardware and software (e.g., as a type 1 cryptographicdevice), as a storage gateway, as a web service, a firewall, ahigh-assurance guard, a server, virtual machine image, one or moredongles, a data warehousing solution or database service box, or thelike.

FIG. 3 is a logical block diagram of a shippable storage device 100,according to some embodiments. FIG. 3 illustrates various components andmodules of a shippable storage device 100. The device may be configuredwith fewer or additional components or modules. Some component or modulemay be replaced by other component or modules. For example, theprocessor 310 and memory 312 may be replaced by firmware, inembodiments. Various components or modules may perform some or all ofthe processes illustrated in the FIGs., in embodiments.

In FIG. 3, device 100 is illustrated with display 214, network interface306 and persistent storage 350. In the illustrated embodiment, displaydriver 302 provides an interface function between a processor 310 anddisplay 214. For example, to instruct the display to display an address,processor 310 executes computer instructions from memory 312 that sendmessages to display driver 302 that are interpreted by the displaydriver and cause the display driver to display the address on display214.

Network interface 306 acts as an interface between an external network(e.g., a customer network or a service provider network or network 106)and the device. In embodiments, the network interface is configured totransmit instructions to the device or to transmit encrypted data to thepersistent storage 350. Wireless interface 308 may be configured toreceive (e.g., via cellular or Wi-Fi network) instructions from theservice provider. For example, the service provider 120 may send updatedaddress information to the shippable storage device 100 via a cellularnetwork such that the displayed address of the device is updated enroute, thereby changing the destination for the device in-flight suchthat the device is shipped to the updated address instead of the prioraddress.

Input/Output (I/O) interface 304 may be configured to coordinate I/Otraffic between processor 310, memory 312, the display driver, networkinterface 306, wireless interface 308, sensor interface(s) 320 andpersistent storage 350 or peripheral interface. In some embodiments, I/Ointerface 304 may perform any necessary protocol, timing or other datatransformations to convert data signals from one component (e.g., systemmemory 312) into a format suitable for use by another component (e.g.,processor 310). In some embodiments, I/O interface 304 may includesupport for devices attached through various types of peripheral buses,such as a variant of the Peripheral Component Interconnect (PCI) busstandard or the Universal Serial Bus (USB) standard, for example. Insome embodiments, the function of I/O interface 340 may be split intotwo or more separate components, such as a north bridge and a southbridge, for example. Also, in some embodiments, some or all of thefunctionality of I/O interface 304, such as an interface to systemmemory 312, may be incorporated directly into processor 310.

The shippable storage device 100 is depicted with persistent datastorage 350. Persistent data storage 350 may include any combination ofnon-volatile storage such as hard drives or flash memory. Persistentstorage 350 may be configured (e.g., during a provisioning process) tostore large amounts of encrypted data (e.g., from a large data storesuch as a customer storage system) during shipment from the customerlocation to a service provider location where the data is transferred toa service provider storage system.

The shippable storage device 100 is depicted with power source 330 thatmay power the various electronic components of the shippable storagedevice 100 and with sensor(s) 340 and sensor interface(s) 320. Asdescribed above, any of various sensor(s) may be incorporated intodevice 100. Device 100 may also include various sensor interface(s) 320that act as an interface between the sensor(s) 340 and I/O interface304. The sensor interfaces may be proprietary interfaces, customized fora particular sensor, in embodiments. The sensor interfaces may performvarious functions such as conversions of data, analysis of sensor outputand output of information based on the analysis or the like.

The shippable storage device 100 is also depicted with a trustedplatform module (TPM) 360. The TPM 360 may provide additional securityfeatures for the shippable storage device 100. For example, after thestorage service provider 102 receives a TPM 360 from a customer, thestorage service provider 102 may communicate with the TPM 360 todetermine whether a change has been made to the configuration of theshippable storage device 100. Changes to the shippable storage device100 configuration may indicate that the shippable storage device 100 wastampered with and that a third party may have accessed data on theshippable storage device 100.

Data Transfer Device Lifecycle

FIG. 4 is a flow diagram of a process of securely transferring datausing a shippable storage device 100, according to some embodiments. Theillustrated process may be performed within the context of a shippablestorage device 100, storage service provider 102 and customer. Theprocess is shown as a data transfer lifecycle for a shippable storagedevice 100.

At 402, a customer creates a data import job for importing data into astorage service provider 102. The customer may create the job through aconsole interface of a computing device that provides communication withthe storage service provider 102 over a network 106. The storage serviceprovider 102 may then provision a shippable storage device 100 and setan electronic display 108 of the shippable storage device 100. Forexample, the storage service provider 102 may set the electronic display108 (e.g., via the display 108 user interface or via an externalconnection through the network interface) to display a customerdestination address. In some embodiments, the storage service provider100 may also store a return address or the address of another customerfacility in memory of the shippable storage device 100 such that thedisplay 108 can be updated with the stored address at some point laterin the data transfer device lifecycle.

The storage service provider 102 then ships the shippable storage device100 to the customer. In some embodiments, the shippable storage device100 may be shipped as a self-contained shipping container to adestination that is indicated by the device's electronic display 108.For example, the storage service provider may provide the shippablestorage device 100 with the enclosure, the display 108, the persistentstorage and the network interface to a common carrier without anyadditional packaging or labeling. The common carrier may ship the devicethrough the carrier network to the destination without any packaging orlabeling in addition to the enclosure and electronic display, inembodiments. In some embodiments, the shippable storage device 100 hasno display 108, and therefore a label and/or packaging is required todisplay the destination address.

At 404, the customer obtains and installs a data transfer tool that isconfigured to encrypt and transfer the customer data to the shippablestorage device 100. For example, the customer may download the datatransfer tool. In some embodiments, the data transfer tool is stored ona hardware storage device, such as a CD or other persistent storagemedium, and received by the customer. In some cases, the customer mayalready have the data transfer tool installed. For example, the customermay have used the data transfer tool for one or more import jobs in thepast.

At 406, the customer receives the shippable storage device 100. Thedevice is installed onto a network at the customer site. Theinstallation may include several steps, described in more detail below.The customer also downloads a job manifest. In some embodiments, the jobmanifest is obtained separately from the data transfer tool. They may bedownloaded in different communication sessions and/or through differentcommunication pathways. For example, the job manifest may be sent viaemail or on a separate device, such as a USB key. The job manifestincludes encryption keys and metadata associated with the job. Theencryption keys may be used to encrypt customer data before the customerdata is stored onto the shippable storage device 100. The metadata mayinclude identification information for the data import job, theshippable storage device 100, and encryption keys. The job manifestitself may also be encrypted. The encryption key to decrypt the jobmanifest can be delivered in the same or alternate communication path,as is done with the job manifest itself.

At 408, the data transfer tool encrypts customer data and transfers theencrypted customer data to the shippable storage device 100. The datatransfer tool may generate encryption keys to encrypt the customer data.The data transfer may also use encryption keys from obtained from thejob manifest to encrypt the customer data.

The electronically displayed destination may then be updated and theshippable storage device 100 shipped as a self-contained shippingcontainer to the updated destination indicated by the device'selectronic display. The display may be updated with a destinationaddress or code that was stored in memory of the shippable storagedevice 100 at provisioning or received over a network (network 106)while en route or at the customer location. The updated address may be areturn address for returning the device to the storage service provider102 or an address of another location for the same or different customer(e.g., security provisions may be implemented such that data frommultiple customers can be stored on the device, so that the device isshipped to other locations before finally being shipped back to thestorage service provider 102). The shippable storage device 100 isshipped to the storage service provider 102. In some embodiments, theshippable storage device 100 can be shipped to one or more othercustomer locations until the shippable storage device 100 is updatedwith the storage service provider 102 address and given to a carrier toship back to the storage service provider 102.

At 410, the encrypted data from the shippable storage device 100 isingested at the storage service provider 102. For example, the shippablestorage device 100 is received by the service provider 102, connected toa service provider network, the data from the shippable storage device100 is decrypted, and the decrypted data is stored at one or morestorage locations of the storage service provider 102. At 412, theshippable storage device 100 is wiped of data (e.g., customer data andsecurity information deleted or overwritten) and prepared for reuse. Theprocess may begin again at 402.

Secure Data Transfer Scheme

FIG. 5 is a logical block diagram illustrating a secure data transferscheme using a shippable storage device 100, according to someembodiments. The storage service provider 102 may be set up by an entitysuch as a company or a public sector organization to provide one or moreservices (such as various types of cloud-based storage and computing)accessible via the Internet and/or other networks to a customer. Thestorage service provider 102 may include numerous data centers hostingvarious resource pools, such as collections of physical and/orvirtualized computer servers, storage devices, networking equipment andthe like, needed to implement and distribute the infrastructure andservices offered by the storage service provider 102. In embodiments,storage service provider 102 may provide various storage services, suchas storing or transferring at least some of the data 500 for a customerin storage device 502 of the storage service provider 102. In someembodiments, the customer data 500 that is transferred to the storagedevice 502 may be organized into one or more different logical orphysical locations (e.g., buckets) within storage device 502, where eachbucket stores one or multiple data objects or files.

A customer may submit a request via a console interface and/orprogrammatic application of a customer device 504 to the storage serviceprovider 102 to create a data import job for importing at least some ofthe data 500 to the storage service provider 120. The customer device504 may be a computing device that provides a user interface and/orapplication that allows the customer to submit the job request to thestorage service provider 102 (e.g., via the network 106). In someembodiments, the customer provides information for the data import job,such as one or more locations at the storage service provider 102 tostore customer data 500 (e.g., one or more buckets within the storagedevice 502). The customer may also indicate a role to assign to thestorage service provider 102 (e.g., read/write and other permissionsassociated with importing the data 500). In some instances, the customermay also select one or more encryption keys to use for encrypting datafor the import job. For example, the customer may select one or moreencryption keys stored at the storage service provider 102 that belongto or are assigned to the customer. In an embodiment, the encryptionkeys are stored in data storage of the storage service provider 102,such as in key data 506 or metadata 508. In some embodiments, thecustomer may instead or additionally request that one or more newencryption keys be generated by the storage service provider 102 for thedata import job.

In the example embodiment, the customer device 504 communicates with theconsole back end 510 of the storage service provider 102. The consoleback end 510 may be a service capable of communicating with the customerdevice 504 as well as other services of the storage service provider102, such as the metadata service 512. In some instances, the consoleback end 510 receives the data import job information described aboveand sends at least some of the job information to the metadata service512 for processing.

The metadata service 512 may supply at least some of the job informationto the job orchestrator 514, which in turn may start the process ofprovisioning a shippable storage device 100 for a new import job. Forexample, the job orchestrator 514 may write information to a shippablestorage device 516 a or cause other services to write information to theshippable storage device 516 a that prepares the shippable storagedevice 516 a for secure data transfer. In some embodiments, a provisionand ingestion service 520 or other service writes information to theshippable storage device 516 a to prepare the shippable storage device516 a for secure data transfer. The provision and ingestion service 520may obtain the information from the job orchestrator 514 and/or themetadata service 512.

In various embodiments, the provision and ingestion service 520represents two or more separate services that each provide differentservices. For example, the provision and ingestion service 520 mayinclude a first service that provides provisioning services beforeshipping a shippable storage device 100 to a customer and a secondservice that provides data ingestion services after receiving theshippable storage device 100 from a customer.

In some embodiments, the information written the shippable storagedevice 516 a may include security information such as one or moreencryption keys or certificates, address information, and/or otherdevice-related information. After the shippable storage device 516 a isprovisioned with the information, the display 108 may be updated todisplay the address of the customer that requested the data import job.The storage service provider 102 may then ship the shippable storagedevice 516 a to the customer.

In some instances, the customer installs a data transfer tool 522 ontothe customer network 104. The data transfer tool 522 is an applicationthat encrypts customer data 500 and transfers the encrypted customerdata to a shippable storage device 516 b. The data transfer tool mayprovide a user interface (e.g., graphical user interface and/or commandline interface) on a display of a computing device of the customernetwork 104 in order to receive input from a user and to provide output.The shippable storage device 516 b may represent the shippable storagedevice 516 a after is arrives at the customer network 104.

In various embodiments, the data transfer tool 522 includes anencryption server 524 to perform at least some of the encryption of thecustomer data 500. The customer may download the data transfer tool 522from the storage service provider 102 over the network 106. In someembodiments, the customer downloads the data transfer tool 522 via acomputing device of the customer network 104, such as a computing devicethat includes the data transfer tool 522.

In some embodiments, the data transfer tool 522 downloads a manifest andmanifest encryption key associated with a data import job from the datatransfer tool back end 526 of the storage service provider 102 via thenetwork 106. In other embodiments the data transfer tool 522 uses apreviously downloaded manifest and manifest encryption key associatedwith the data import job. The manifest may include informationassociated with a particular data import job that the customer requestedusing the customer device 504. Further, the particular import job may beassociated with the shippable storage device 516 b. For example, theshippable storage device 516 b may have been provisioned and shipped inresponse to the customer submitting a data import job request throughthe customer device 504. In some instances, the manifest may includemetadata associated with the data import job such as a job ID, a deviceID, security information, encryption keys, and locations for storingdata in the storage device 502 (e.g., bucket ID's). In some embodiments,the data transfer tool 522 uses at least some of the information fromthe manifest to process and transfer the customer data 500 to theshippable storage device 516 b.

In some embodiments, the data transfer tool back end 526 may alsoreceive information from the data transfer tool 522. For example, thedata transfer tool back end 526 (or other service) may provideinstructions to the data transfer tool 522 to implement a data transferplan for one or more shippable storage devices 100 connected to thecustomer network 104. For example, the instructions may coordinate whichportions of the customer data 500 are copied onto correspondingshippable storage devices 100, depending on one or more characteristicsof the customer network or data 500 (e.g., transfer speeds for eachconnection with each shippable storage device 100, format of data 500,characteristics of a storage destination of the data 500). Inembodiments, the data transfer tool back end 526 may manage thegeneration of shards based on redundant data encoding (e.g., erasureencoding, data striping, etc.) for the data 500. Thus, any of theprocesses associated with the data transfer tool 522 may instead becontrolled remotely by the data transfer tool back end 526, or incooperation with the data transfer tool 522. In some embodiments, thedata transfer tool back end 526 monitors data collected by a shippablestorage device 100 attached to the customer network 104 (e.g., tomonitor performance/health of one or more client systems and efficiencyof transferring data to one or more shippable storage devices).

In some embodiments, the data transfer tool 522 may also executealgorithms to develop plans and patterns for data placement on one ormore shippable storage devices 100 connected to the customer network104. For example, the plans and patterns may coordinate which portionsof the customer data 500 are copied onto corresponding shippable storagedevices 100, depending on one or more characteristics of the customernetwork or data 500 (e.g., transfer speeds for each connection with eachshippable storage device 100, format of data 500, characteristics of astorage destination of the data 500). In embodiments, the data transfertool 522 may manage the generation of shards based on redundant dataencoding (e.g., erasure encoding, data striping, etc.) for the data 500.Thus, any of the processes associated with the data transfer tool 522may operate autonomously and not require communication or coordinationwith any other processes, services, or tools.

The storage service provider 102 may also include a data tech console528. The data tech console 528 may be a service that communicates withone or more other services, such as the metadata service 512. In someembodiments, the data tech console provides information to an interface,such as a graphical user interface or command line interface of adisplay, that allows a data technician to view and change informationassociated with the storage service provider 102. For example, the datatech console may provide information regarding the status of variousdata import jobs processed by the storage service provider 102 orprovide data stored on the storage device 502, key data 506, andmetadata 508.

FIGS. 6-26 illustrate various processes and systems associated with theshippable storage device 100. One or more portions of the illustratedprocesses may be performed by one or more processes executing on thestorage service provider 102 and/or the customer network 104, inembodiments (e.g., one or more of the services described in FIG. 5). Insome embodiments, one or more portions of the illustrated process mayassociate with a particular data import job for importing data 500 fromthe customer network 104 to the storage service provider 102. In someinstances, the same or similar processes and systems may be implementedfor the storage service provider 102 and/or the customer network 104 totransfer/export data from the storage service provider 102 to thecustomer network 104. Further, any job-related information generated orprocessed may be stored in one or more locations of the storage serviceprovider 102, such as key data 506, metadata 508, and the storage device502.

FIG. 6 is a flow diagram of a process for collecting information for anew data import job to securely transfer data using a shippable storagedevice 100, according to some embodiments. One or more portions of theillustrated process may be performed via one or more service of thestorage service provider 102, such as by console back end 510.

At block 602, the storage service provider 102 receives logincredentials (e.g., username, password, and/or one or more other securitycodes) from a customer device 504. In some embodiments, a console backend 510 may be configured to receive the login credentials and uponverifying the credentials, authenticate the customer and provide accessand management of data import and/or export jobs through the consoleinterface of the customer device 504. The console interface may providean indication of a status of one or more data import jobs of thecustomer. Thus, the console back end 510 may receive information fromthe customer network 104 and provide information to the customer network104. The storage service provider 102 may use at least some of thereceived information for provisioning a shippable storage device 100 forsecure data transfer.

At block 604, the console back end 510 may receive the request for a newdata import job. In some embodiments, the console back end 510 forwardsthe request to the metadata service 512, which generates informationassociated with the new job, such as a job ID. The console back end 510may then receive the generated information and present at least some ofthe generated information to the customer via the console (e.g.,displaying the job ID for the new data import job).

At block 606, the console back end 510 may determine the customershipping address for shipment of the shippable storage device 100. Insome embodiments, the console back end 510 determines the customeraddress by reading a customer address that is associated with thecustomer and stored on the storage service provider 102. In someinstances, the customer provides the customer address via the customerdevice 504, such as through a user interface.

At block 608, the console back end 510 may determine permissions for theimport job. In some embodiments, the console back end 510 determines thecustomer address by reading the permissions associated with the customerthat are stored on the storage service provider 102. In some instances,the customer provides the permissions via the customer device 504, suchas through a user interface. The permissions may specify reading,writing, or other permissions suitable for associating with one or moreparticular roles for accessing or handling the imported data. Thus, inembodiments, a role is determined for the data import job, where therole is a set of one or more permissions given to the storage serviceprovider 102 and/or other entity.

At block 610, the console back end 510 may determine preferences for theimport job. In some embodiments, the console back end 510 determines atleast some of the preferences by reading preferences associated with thecustomer that are stored on the storage service provider 102. In someinstances, the customer provides the preferences via the customer device504, such as through a user interface. The preferences may specifywhether the customer receives a notification of the status of one ormore corresponding activities associated with the data import job (e.g.,data import progress, data import completed, etc.) and how the customerreceives notifications (email, SMS text message, phone call, etc.).

At block 612, the console back end 510 determines a location to storedata the imported data at the storage service provider 102. For example,the console back end 510 receives from the customer device 504 anindication of a location to store data at the storage service provider102 (e.g., a particular type of service or location of the storageservice provider 102). In some embodiments, the console back end 510receives an indication of multiple available locations to store customerdata 500 at the storage service provider 102. In some embodiments, theconsole back end 510 receives one or more location ID's (e.g., bucketID's) from the metadata service 512 that correspond to one or moreavailable locations to store data. The console back end 510 may thenreceive the location ID's and provide them to the customer device 504for display. The console back end 510 may then receive an indication ofone or more of the location ID's from the customer.

At block 614, the console back end 510 determines one or more encryptionkeys for the data import job. In some embodiments, the console back end510 may select one or more available encryption keys stored in key data506. In some instances, the console back end 510 may receive from thecustomer device 504 a selection of one or more available encryption keysstored in key data 506. In some embodiments, the console back end 510generates one or more of the encryption keys. In various embodiments, acombination of selected keys and generated keys may be used.

At block 616, the storage service provider 102 stores information forthe data import job, including any of the information received ordetermined at blocks 602-614. In some embodiments, the console back end510 saves the information in key data 506 and/or metadata 508.

FIG. 7 is a flow diagram of a process for provisioning a shippablestorage device 100 to securely transfer data for a data import job,according to some embodiments. One or more portions of the illustratedprocess may be performed via the storage service provider 102, such asby the provision and ingestion service 520 and/or other services.

At block 702, the storage service provider 102 writes operating softwareto the shippable storage device 100. For example, a provisioning servicemay write an operating system and servers to the shippable storagedevice 100. One of the servers may allow a device of the customernetwork 104 to communicate with the shippable storage device 100 througha command line or graphical user interface (e.g., to communicate withthe data transfer tool 522). In some embodiments, the provisioningservice may also write a display server to the shippable storage device100 that sends information to the display 108 (e.g., to display ashipping address).

At block 704, the storage service provider 102 writes securityinformation to the shippable storage device 100. For example, aprovisioning service may write a security certificate or rootcertificate to the shippable storage device 100 that allows a device ofthe customer network 104 to authenticate the shippable storage device100 based on receiving the security information from the shippablestorage device 100 and processing the received security information. Thesecurity information may also allow the shippable storage device 100 toauthenticate the device of the customer network 104 based on receivingthe security information from the device of the customer network 104 andprocessing the received security information. Thus, the securityinformation written to the shippable storage device 100 may be used in amutual authentication processes between the shippable storage device 100and a device of the customer network 104.

At block 706, the storage service provider 102 writes shippinginformation to the shippable storage device 100. For example, aprovisioning service may write a shipping address of the customer and ashipping address of the storage service provider 102 to the shippablestorage device 100. The shippable storage device 100 may displaydifferent destination shipping addresses for the shippable storagedevice 100, depending on what part of the data transfer cycle theshippable storage device 100 is on. In some embodiments, one or more ofthe shipping addresses may be updated by the storage service provider102 or the customer if either determines that the shipping address haschanged.

At block 708, the storage service provider 102 indicates that theshippable storage device 100 is provisioned and ready for shipment tothe customer. For example, a provisioning service may store or updateinformation in metadata 508 that indicates the shippable storage device100 is provisioned and ready for shipment. In some embodiments, theprovisioning service may generate and send a message to another serviceor device of the storage service provider 102 to indicate that theshippable storage device 100 is provisioned and ready for shipment.

FIG. 8 is a flow diagram of a process for sending a shippable storagedevice 100 and a manifest to a customer, according to some embodiments.One or more portions of the illustrated process may be performed via thestorage service provider 102, such as by the job orchestrator 514, theprovision and ingestion service 520, and/or other services.

At block 802, the storage service provider 102 sends the shippablestorage device 100 to the customer. For example, the shippable storagedevice 100 is sent to a location of the customer network 104. Theshippable storage device 100 may be sent via one or more suitable formsof transportation.

At block 804, the storage service provider 102 sends the data transfertool 522 to the customer. In some embodiments, the storage serviceprovider 102 sends the data transfer tool 522 to a device of thecustomer network 104 via the network 106. For example, the data transfertool 522 may be sent via email attachment, via an internet session, orany other suitable means for sending via the network 106. In someinstances, the data transfer tool 522 may be stored on a storage deviceand shipped to the customer. After the customer obtains the datatransfer tool 522, the customer may install the data transfer tool 522on the customer network 104.

At block 806, the storage service provider 102 generates a manifestbased on information for a data import job. In some embodiments, thestorage service provider 102 creates one or more files that containsecurity information, one or more encryption keys, and metadataassociated with the data import job. The security information mayinclude information for authenticating the shippable storage device 100and/or allowing the shippable storage device 100 to authenticate adevice of the customer network 104. The metadata may include informationthat identifies one or more data storage locations in storage device502. In some embodiments, the metadata may associate one or more of thedata storage locations with one or more of the encryption keys.

At block 808, the storage service provider 102 sends the manifest to thecustomer. In some instances, the storage service provider 102 sends themanifest to the customer network 104 in response to determining that theshippable storage device 100 is attached to the customer network. Insome embodiments, the storage service provider 102 encrypts the manifestwith an encryption key before sending it to the customer network 104.For example, the storage service provider 102 may use an encryption keyfrom key data 506 that is assigned to the customer or generate anencryption key to encrypt the manifest. In some embodiments, themanifest may be sent via email attachment, via an internet communicationsession, or any other suitable means for sending via the network 106. Insome instances, the manifest may be stored on a storage device andshipped to the customer. After the customer obtains the manifest, thecustomer may decrypt the manifest via the data transfer tool 522, if itis encrypted. In some embodiments, the customer enters a code (e.g.,numbers, letters, and/or symbols) and if the code is correct, the datatransfer tool 522 decrypts the manifest. In some instances, the customeruses an encryption key that may be stored within the customer network104 to decrypt the manifest.

At block 810, the storage service provider 102 sends a manifest code foraccessing the manifest to the customer. In some embodiments, themanifest code may be sent via email attachment, via an internet session,or any other suitable means for sending via the network 106. In someinstances, the manifest code may be stored on a storage device andshipped to the customer. After the customer obtains the manifest code,the customer may access the manifest by entering the manifest code intothe data transfer tool 522. In some embodiments, at least two of thedata transfer tool, the manifest, and the manifest code are sentseparately.

FIG. 9 is a logical block diagram of a shippable storage device 100ready for shipping to a customer, according to some embodiments. Theshippable storage device 100 may include a persistent data storage 902.The persistent data storage 902 may include information associated withthe data import job. In some embodiments, the information may includeaddress information 904, such as a shipping address of the storageservice provider 102 and/or the shipping address associated with thecustomer network 104. The address information may store the nextshipping address after the customer address and/or one or more otherintermediate destination addresses for the shippable storage device 100.In some embodiments, the address information 904 includes one or moreshipping labels, wherein each shipping label corresponds to a particulardestination address. The shipping label may include any informationnecessary to display a shipping label on the display 108, including atleast any information that a conventional printing label would provide.Further, the information may include security information 906, such assecurity certificates and/or encryption keys. In some instances, theinformation may include a device ID 704 to uniquely identify theshippable storage device 100 and/or an import ID to identify the dataimport job associated with the shippable storage device 100.

In various embodiments, the display 108 may display a customer address908 of the customer. For example, the customer address 908 may be theshipping address associated with the customer network 104 when theshippable storage device 100 is being shipped to the customer.Conversely, when the shippable storage device 100 is being shipped fromthe customer back to the storage service provider 102, the shippingaddress of the storage service provider 102 may be displayed by thedisplay 108. In some instances, at least some of the address information904 and the security information 906 stored in the persistent datastorage 702 may be provided by the storage service provider 102 when theshippable storage device 100 is provisioned.

FIG. 10 is a flow diagram of a process for receiving a shippable storagedevice 100 and information for a data import job, according to someembodiments. One or more portions of the illustrated process may beperformed via the customer, such as by using the data transfer tool 522.

At block 1002, the customer downloads the data transfer tool 522 fromthe storage service provider 102 and installs the data transfer tool 522on a computer system at the customer network 104. In some embodiments,the customer may use the customer device 504 or other computing deviceto download the data transfer tool 522 from the storage service provider102. Before downloading the data transfer tool 522, the customer may berequired to enter credentials that are verified by the storage serviceprovider 102. In some instances, the data transfer tool 522 may alreadybe installed at the customer location. For example, the data transfertool 522 may have been used for a previous data import job.

At block 1004, the customer receives the shippable storage device 100and connects the shippable storage device 100 to the customer network104. In some embodiments, one or more security steps may be performed toauthenticate the shippable storage device 100 before connecting theshippable storage device 100 to the customer network. For example, oneor more bar codes may be scanned and RFID chips may be read. In variousembodiments, any other suitable verification techniques may be used toauthenticate the shippable storage device 100 before connecting theshippable storage device 100 to the customer network 104. The customerthen connects the shippable storage device 100 to the customer networkvia the network interface 240 of the device. In some embodiments, theshippable storage device 100 has multiple network interfaces that areeach a different type of network interface. In some instances, there aretwo or more network interfaces of a particular type. The customer mayselect a network interface to connect the shippable storage device 100to the customer network 104.

At block 1006, in response to being connected to the customer network104, the shippable storage device 100 and/or the data transfer tool 522determines whether user configuration is needed for the shippablestorage device 100. In some embodiments, the shippable storage device100 and/or the data transfer tool 522 determines that user configurationis required to select one or more network-related configuration, such asan IP address, a type of network interface used (e.g., optical, SPF), anetmask, and a gateway. In some embodiments, in response to determiningthat user configuration is required for the shippable storage device100, the process continues to block 1008. In some embodiments, a userconfiguration will be needed if the user decides to override one or moredefault settings for the network parameters or other parameters relatedto the shippable storage device 100 and/or the customer network 104.

At block 1008, the display 108 of the shippable storage device 100 mayprovide a user interface that permits a user to enter one or moreconfiguration parameters for the shippable storage device 100, such asIP address, a type of network interface used (e.g., optical, SPF), anetmask, and a gateway. The display 108 may provide multipletouch-enabled keys that allow the user to enter one or more parameters.In various embodiments, other suitable techniques may be implemented toallow a user to enter parameters for the shippable storage device 100.The process then continues to block 1010.

At block 1006, if the shippable storage device 100 determines that userconfiguration is not needed for the shippable storage device 100, theprocess continues to block 1010. At block 1010, the customer downloads amanifest and a manifest code from the storage service provider 102. Insome embodiments, the customer downloads the manifest and manifest codefrom the data transfer tool back end 526 of the storage service provider102.

In some instances, the customer may download the manifest and/ormanifest code using a different pathway or application, such as via thecustomer device 504 or other computing system of the customer network104. For example, the manifest may be downloaded using a different pathor application than that of the manifest code. In some embodiments, themanifest may be downloaded using the same path, but at a different pointin time or via a separate session or transaction. Further, in someinstances, the manifest code may be delivered via email, text message,physical mail, or any other suitable form for communicating a code. Invarious embodiments, the manifest code may be composed of one or morenumbers, letters, and/or symbols.

At block 1012, the customer provides the manifest file location and theIP address of the shippable storage device 100 to the data transfer tool522. For example, the customer may provide a directory path and filename of the manifest file to the data transfer tool 522. In variousembodiments, other suitable identifiers for the location of the manifestfile may be provided to the data transfer tool 522. Further, in someinstances, the manifest file location and/or the IP address of theshippable storage device 100 may be automatically detected or stored inthe data transfer tool 522, eliminating the need for a user to entereither or both.

At block 1014, the customer provides the manifest code to the datatransfer tool 522. In some embodiments, the customer may provide themanifest code, the manifest file location, and/or the IP address of theshippable storage device 100 at substantially the same time (e.g., usingthe same screen of a user interface or a series of associated userinterfaces). In some instances, in response to providing the manifestcode to the data transfer tool 522, the data transfer tool 522 decryptsthe manifest using the manifest code and/or a customer-assigned key andprovides access to the information in the manifest, including encryptionkeys. The data transfer tool may initiate the process of identifyingcustomer data 500, encrypting the identified customer data 500 and/ortransferring the encrypted customer data 500 to the shippable storagedevice 100. In some instances, in response to providing the manifestcode, the data transfer tool 522 may determine whether the manifest codeis correct. If it is not correct, the data transfer tool 522 may providea message indicating that an incorrect code was provided and prompt fora user to enter another manifest code. In various embodiments, inresponse to validating the manifest code, the data transfer tool 522accesses and/or obtains one or more encryption keys of the manifest.

FIG. 11 is a logical block diagram of shippable storage devices 100connected to a customer network 104, according to some embodiments. Insome embodiments, multiple shippable storage devices 100 a-100 n may beconnected to the customer network 104. For example, each of theshippable storage devices 100 may communicate with a correspondingcomputing device 1102 that includes a memory 1104 that further includesthe data transfer tool 522 and the encryption server 524. Further, eachcomputing device 1102 may communicate with one or more data storagedevices that include the customer data 500 to be imported from thecustomer to the corresponding data storage service 102. In someembodiments, multiple data import jobs are requested by the customer,each import job associated with a different one of the shippable storagedevices 100. The shipping devices associated 100 with the multiple dataimport jobs may be connected to the customer network 104 concurrentlyand download encrypted data concurrently.

In various embodiments, a particular memory 1104 of a computing device1102 may implement multiple instances of the data transfer tool 522.Furthermore, two or more of the multiple instances may communicate withone particular shippable storage device. In some embodiments, aparticular instance of the data transfer tool 522 may communicate withmultiple shippable storage devices. Thus, the relationship betweeninteraction between data transfer tools 522 and the shippable storagedevices 100 may be one-to-one, many-to-one, one-to many, ormany-to-many. The above relationships may also apply whether the datatransfer tools 522 are implemented on one computing device 1102 oracross multiple computing devices 1102.

In some instances, in response to the storage service provider 102determining that multiple shippable storage devices 100 will be requiredfor an amount of data for a particular data import job request, thestorage service provider 102 will create two or more new data importjobs, wherein each of the new data import jobs corresponds to adifferent portion of the customer data 500 to be imported. Further, eachof the new data import jobs may correspond to a different shippablestorage device 100.

In some embodiments, the display 108 may be capable of receiving inputvia touch. The display 108 may display one or more graphical elementsthat are associated with one or more corresponding options for a user toenter information associated with the shippable storage device 100. Forexample, a first button 1106 may be selectable by a user for entering aninternet protocol (IP) address for the shippable storage device 100.After selecting the first button 1102 via touch, the user may bepresented multiple touch-enabled keys that allow the user to enter an IPaddress. In some embodiments, an IP address is automatically assigned tothe shippable storage device 100 after the shippable storage device 100is connect to a network of the customer (e.g., via DHCP). The datatransfer tool 522 may use the IP address of the shippable storage device100 in order to establish communication with the shippable storagedevice 100.

In some embodiments, the display 108 may display a second button 1108that is selectable by a user for entering a type of network interfacefor the shippable storage device 100. For example, after selecting thesecond button 1108 via touch, the user may be presented with two or moretouch enabled buttons, each button representing a different type ofnetwork interface (e.g., RJ45, SPF, optical). Any other suitable optionsfor a network interface type may be available for selection by the user,in embodiments.

FIG. 12 is a flow diagram of a process for decrypting a manifest andauthenticating a shippable storage device, according to someembodiments. One or more portions of the illustrated process may beperformed via the customer, such as by using the data transfer tool 522.

At block 1202, the data transfer tool 522 decrypts the manifest usingthe manifest code. In some embodiments, the manifest is decrypted by akey that is provided by the customer to the data transfer tool 522. Insome instances, the key is downloaded from the storage service provider102 separately from the manifest (e.g., a separate communication sessionor downloaded by a different computing device of the customer network104). When a user provides the key to the data transfer tool 522 (e.g.,via a user interface) the data transfer tool 522 may decrypt themanifest and access the encryption keys, security information, and/orother data associated with the data import job.

At block 1204, the data transfer tool 522 discovers the shippablestorage device 100 on the customer network 104. The data transfer tool522 may discover the shippable storage device 100 based at least on theIP address of the shippable storage device 100. The IP address may beassigned in various ways, such as those described in FIG. 10.

At block 1206, the data transfer tool 522 authenticates the shippablestorage device 100 with the security information obtained from themanifest. For example, the data transfer tool 522 may read a rootcertificate or encryption key from the shippable storage device 100 andprocess the certificate or encryption key with the security informationfrom the manifest to authenticate the device (e.g., verify that theshippable storage device 100 is the same device that was shipped fromthe storage service provider 102). In some embodiments, the shippablestorage device 100 also authenticates the data transfer tool 522 and thecustomer network 104 based on reading a root certificate or encryptionkey from the data transfer tool 522 and processing the certificate orencryption key with the security information on the shippable storagedevice 100 to authenticate the data transfer tool 522 and/or thecomputing device 1102 of the customer network 104. Thus, the datatransfer tool 522 and the shippable storage device 100 may mutuallyauthenticate each other.

At block 1208, the data transfer tool 522 generates a virtual filesystem on the customer network for the data transfer tool to processdata. In some embodiments, the virtual file system includes volatileand/or non-volatile memory configured to store a copy of at least someof the customer data 500 and to store processing results of the customerdata. For example, the data transfer tool 522 may encrypt data stored inthe virtual file system and store the encrypted data in the virtual filesystem. In some embodiments, the encrypted data is then copied from thevirtual file system to the shippable storage device 100.

FIG. 13 is a logical block diagram of a manifest 1300 includinginformation associated with a data import job, according to someembodiments. In some embodiments, the manifest 1300 is downloaded by thecustomer using the data transfer tool 522. If the customer has not yetinstalled the data transfer tool 522, then the customer may need toinstall it before downloading the manifest 1300.

In some embodiments, the data transfer tool 522 communicates with aservice of the data storage provider 102, such as the data transfer toolback end 526, in order to download the manifest 1300. The customer maybe required to enter credentials and/or a job ID in order for the datatransfer tool 522 to download the manifest 902. In some instances, theshippable storage device 100 associated with the data import job mayfirst need to be connected to a network of the customer 100 and detectedby the data transfer tool 522 and/or the storage service provider 102before the manifest 1300 can be downloaded. Therefore, the storageservice provider 102 may send the manifest 1300 for a particular dataimport job to the data transfer tool 522 in response to determining thatthe shippable storage device 100 attached to the customer network 104 isassociated with the particular data import job (e.g., byobtaining/verifying security information from the device, such as via aroot certificate).

The manifest 1300 may include various types of information associatedwith the import job, at least some of which is used to encrypt and/ortransfer data to the shippable storage device 100. In some embodiments,the manifest 1300 includes security information 1302 that the customeruses to authenticate the shippable storage device 100. For example, thedata transfer tool 522 may use a certificate or key from the manifest1300 to authenticate the shippable storage device 100 based on obtainingsecurity information from the shippable storage device 100.

In some embodiments, the manifest 1300 includes one or more encryptionkeys 1304 that are used to encrypt keys and/or data before the encryptedkeys and/or data are transferred to the shippable storage device 100 orsent to the storage service provider via a network 106. The encryptionkeys 1304 may be associated with the customer and stored at one or moresecure locations of the storage service provider 102, such as key data506 or metadata 508. One or more of the encryption keys 1304 may haveexisted before the import job was created. For example, the encryptionkeys 1304 may be stored at key data 506 and associated with thecustomer. In some instances, one or more of the encryption keys 1304 mayhave been generated in response to the request for the import job andassociated with the customer.

One or more of the encryption keys 1304 may be used for encrypting oneor more other encryption keys that are generated by the data transfertool 522, before being stored on the shippable storage device 100.Different encryption keys 1304 may be used to encrypt different subsetsof encryption keys before being transferred to the shippable storagedevice 100. For example, each of the encryption keys 1304 a-n mayencrypt a different subset of encryption keys generated by the datatransfer tool 522.

In some embodiments, each of the encryption keys 1304 may correspond toa different storage location of the storage service provider 102. Forexample, the encryption key 1304 a may correspond to a first bucket ofthe storage device 502 of the storage service provider 102 and theencryption key 1304 n may correspond to a second bucket of the storagedevice 502. Thus, any subset of the data 500 that is associated with akey encrypted by the encryption key 1304 a will eventually be stored inthe first bucket (in decrypted or encrypted form) and any subset of thedata 500 that is associated with a key encrypted by the encryption key1304 n will eventually be stored in the second bucket (in decrypted orencrypted form. Moreover, in some instances, two or more of theencryption keys 1304 may be associated with the same location of thestorage device 502. In some embodiments, the encryption keys 1304 areused to encrypt at least some of customer data 500 instead of or inaddition to encryption keys.

In various embodiments, the manifest 1300 includes job metadata 1306,which includes information associated with the data import job and/orthe shippable storage device 100 being used for the data import job. Thejob metadata 1306 may include information identifying one or morelocations at the storage service provider 102 at which differentportions of customer data 500 are to be stored, such as bucket ID 1308and bucket ID 1310. Furthermore, each location identifier may beassociated with one or more encryption keys 1304. For example, bucket ID1308 may be associated with one or more encryption keys 1304 and bucketID 1310 may be associated with one or more other encryption keys 1304.

Therefore, in some embodiments, portions of customer data 500 that areassociated with encryption key 1304 a will be stored in the bucket ofstorage device 502 that corresponds to bucket ID 1308 and portions ofcustomer data 500 that are associated with encryption key 1304 n will bestored in the bucket of storage device 502 that corresponds to bucket ID1310. Although the example embodiment implements two bucket ID's and twocorresponding buckets, any other number of bucket ID's and buckets maybe implemented. Further, any other suitable mapping scheme may beimplemented for associating one or more of the encryption keys 1304 to acorresponding one or more storage locations within the storage device502 of the storage service provider.

In various embodiments, the job metadata 1306 may include additionalinformation associated with a particular data import job. For example,the job metadata 1306 may include a device ID or a job ID thatidentifies the particular data import job associated with importingcustomer data 500 using the shippable storage device 100.

FIG. 14 is a flow diagram of a process for storing encrypted data onto ashippable storage device, according to some embodiments. One or moreportions of the illustrated process may be performed by the datatransfer tool 522.

At block 1402, the data transfer tool 522 identifies data 500 on thecustomer network 104 to be imported into the storage service provider102. In some embodiments, the data transfer tool 522 may receive anindication of the data 500 to be imported, such as by a user interfaceassociated with the data transfer tool 522. In some instances, the datatransfer tool 522 determines at least some of the data 500 to beimported based on information in the job manifest 1300.

At block 1404, the data transfer tool 522 obtains the identified data500 for processing. For example, the data transfer tool 522 may copy theidentified data 500 into one or more portions of memory associated withthe data transfer tool 522. In some embodiments, the data transfer tool522 copies the identified data to a file system or virtual file systemassociated with the customer network 104. In various embodiments, thedata transfer tool 522 may copy the identified data 500 to any type ofstorage location or file system suitable for storing, encrypting, andprocessing the identified data 500. At least some of the data 500 may bestored in the memory 1104 and/or other storage devices associated withthe computing device 1102. Thus, the data transfer tool 522 may use anycombination of volatile and/or non-volatile memory capable of storingthe processing results of the data transfer tool, receiving copies ofthe data 500, and transferring encrypted data and keys onto theshippable storage device 100.

At block 1406, the data transfer tool 522 encrypts the identified data500. In some embodiments, data transfer tool 522 generates encryptionkeys for encrypting the identified data 500 and/or the generatedencryption keys. In some instances, the data transfer tool 522 encryptsat least some of the identified data 500 and/or encryption keys usingencryption keys obtained from the manifest 1300. At block 1408, the datatransfer tool 522 stores (e.g., transfers) the encrypted data andencrypted keys onto the shippable storage device 100.

In some embodiments, the data transfer tool 522 communicates via an API(application programming interface) on the shippable storage device 100in order to transfer encrypted data onto the shippable storage deviceand perform other functions. Thus, data may be pulled from the datatransfer tool 522 or pushed to the tool. In some instances, a virtualfile system is mounted on the computing device 1102 that hosts the datatransfer tool 522. Standard system copying commands may then be used totransfer files to the virtual file system and onto the shippable storagedevice 100 (thus, pushing data to the tool). In various embodiments, thedata transfer tool 522 may provide its own API so that API calls may bemade to the data transfer tool 522 (e.g., a put command that pushes datato the tool).

FIG. 15 is a flow diagram of a process for encrypting data and keys andstoring the encrypted data and encrypted keys onto a shippable storagedevice 100, according to some embodiments. One or more portions of theillustrated process may be performed via the data transfer tool 522. Invarious embodiments, the data transfer tool 522 generates encryptionkeys using any suitable technique for generating keys for encryptingdata.

At block 1502, the data transfer tool 522 generates a file key for eachfile of data 500 to be transferred onto the shippable storage device100. In embodiments, the file key generated for a particular file may beused for encrypting other encryption keys (e.g., chunk keys) thatencrypt the actual data of the particular file. In other embodiments,the file key generated for a particular file is used to encrypt the dataof the file. In some instances, each file of the data 500 is a portionof the data 500 identified by the data transfer tool 522 as capable ofbeing partitioned into two or more chunks of data. In some embodiments,a file of the data 500 may be an object that includes a file of data andadditional metadata that describes one or more characteristicsassociated with the file (e.g., a path associated with the file, filesize, file type).

In some embodiments, each file key may be unique with respect to thefile keys generated for the files in the virtual file system. In someinstances, the same file key may be used for encrypting chunk keysand/or data for two or more files. For example, a first file key may begenerated for use with a first group of files, a second file key may begenerated for use with a second group of files, and so on. In this way,processing time may be reduced and fewer processing resources may berequired to generate keys.

At block 1504, the data transfer tool 522 partitions each file intomultiple data chunks. Thus, in some embodiments, each file may beseparated into two or more chunks of data. In some instances, each chunkof data for a particular file may be the same size or approximately thesame size. In some embodiments, metadata may be added to each chunk(e.g., the file the chunk is from, a chunk key or chunk key IDassociated with the chunk, a file key or file key ID associated with thechunk, chunk order, and/or sequence number for assembling back into afile at a later point).

At block 1506, the data transfer tool 522 generates a chunk key for eachchunk of the data to be transferred onto the shippable storage device100. In embodiments, each chunk key encrypts the corresponding chunkbefore the chunk is transferred to the shippable storage device 100.

In some embodiments, each chunk key may be unique with respect to thechunk keys generated for the files in the virtual file system. In someinstances, the same chunk key may be used for two or more chunks from aparticular file. For example, a first chunk key may be generated for afirst group of chunks from a particular file, a second chunk key may begenerated for a second group of chunks from the particular file, etc. Insome embodiments, one or more chunk keys that are generated for a firstfile may be also be used for one or more other files. Using some of theabove techniques, processing time may be reduced and fewer processingresources may be required to because fewer chunk key are generated.

At block 1508, the data transfer tool 522 encrypts the chunks using thechunk keys. In some embodiments, each chunk key is assigned to acorresponding chunk and encrypts the corresponding chunk. In someembodiments, a particular chunk key may encrypt two or more chunks.

At block 1510, the data transfer tool 522 encrypts the chunk keys usingthe file keys. Thus, each file key may encrypt the multiple chunk keysthat are associated with the corresponding file. In some embodiments, ifthe data transfer tool 522 determines that the size of a particular fileis below a threshold size, then the data transfer tool 522 encrypts theencrypted chunks associated with the particular file key. In suchinstances, the data transfer tool 522 may or may not also encrypt thechunk keys associated with the particular file using the associated filekey.

At block 1512, the data transfer tool 522 encrypts the file keys usingthe bucket keys obtained from the manifest 1300. In some embodiments,each file is assigned for storage within a corresponding bucket on thestorage device 502 of the storage service provider 102 and thereforeeach file key is assigned to a bucket key associated with the bucket inwhich the corresponding file is to be stored. For example, a first filemay be assigned to be stored in a first bucket that is associated withone or more bucket keys. Therefore, a file key corresponding to thefirst file may be encrypted by one of those one or more bucket keys.

In various embodiments, any number of additional levels of encryptionmay be used to encrypt keys generated by the data transfer tool 522and/or keys obtained from the manifest 1300. For example, the datatransfer tool 522 may generate a first additional level one or more keysthat are used to encrypt the file keys. Then, the bucket keys are usedto encrypt the first additional level of one or more keys instead ofencrypting the file keys. As another example, the data transfer tool 522may generate a second additional level of one or more keys to encryptone or more corresponding keys of the first additional level, whereinthe first additional level of keys encrypts the file keys.

In some embodiments, the bucket keys are securely stored by the storageservice provider 102 and are made temporarily available to the datatransfer tool 522 for encrypting the file keys obtained from themanifest 1300. Thus, in embodiments, the bucket keys are deleted frommemory associated with the data transfer tool and the computing device1102 within a short period of time after the bucket keys are used forencryption. In some embodiments, a bucket key is also used to encryptone or more other portions of data and/or keys. For example, the bucketkeys may be used to encrypt other data, keys, encrypted keys, and/orencrypted chunks.

At block 1514, the data transfer tool 522 transfers the encryptedchunks, the encrypted chunk keys, and the encrypted file keys to theshippable storage device 100. In response to determining that thetransfer is complete, the data transfer tool 522 may then provide anindication that that the shippable storage device 100 is ready forshipment to the storage service provider 102. For example, the datatransfer tool 522 may cause a user interface to display a message thatthe shippable storage device 100 is ready for shipment or may send anotification to one or more services or devices of the customer network104. In various embodiments, file keys are not generated. In such cases,one or more bucket keys may be used to encrypt the chunk keys and thenthe data transfer tool 522 transfers the encrypted chunks to theshippable storage device 100 and may also transfer the encrypted chunkkeys to the shippable storage device 100.

In some embodiments, the data transfer tool 522 does not transfer any ofthe data 500 or keys onto the shippable storage device 100 in anunencrypted form. For example, the data transfer tool 522 may store theencrypted chunks, the encrypted chunk keys, and the encrypted file keyswithout storing the chunks, chunk keys, and file keys in unencryptedform.

In various embodiments, the data transfer tool 522 may transfer theencrypted chunks to the shippable storage device 100 withouttransferring the encrypted chunk keys and the encrypted file keys to theshippable storage device 100. The data transfer tool 522 may then sendthe encrypted chunk keys and the encrypted file keys to the data storageservice provider 102 via the network 106, separate from the shippablestorage device 100. In various embodiments, the data transfer tool 522may send the file keys and/or the chunk keys to the data storage serviceprovider 102 via the network 106 in unencrypted form (but within asecure connection) or in a form encrypted by one or morecustomer-assigned keys that are stored on the storage service provider102.

In some embodiments, the data transfer tool 522 discovers multipleshippable storage devices 100 attached to the customer network 104. Thedata transfer tool 522 may then determine a data transfer plan fortransferring the encrypted chunks, the encrypted chunk keys, and/or theencrypted file keys to the plurality of the shippable storage devices.Based on the data transfer plan, the data transfer tool 522 may transfera different portion of the encrypted chunks, the encrypted chunk keys,and/or the encrypted file keys to each of multiple shippable storagedevices 100. Furthermore, the transferring to two or more of theshippable storage devices 100 may occur in parallel (e.g.,concurrently). The data transfer plan may be based on one or morefactors, including a transfer speed associated with one or more of theshippable storage devices 100, an available storage capacity of one ormore of the shippable storage devices 100, and one or morecharacteristics of the source of at least some of the identified data500 (e.g., type of storage device the data 500 is stored on, a format ofthe data 500), and one or more characteristics of a destination of atleast some of the identified data 500 (e.g., type of storage device thedata 500 will be stored on at the remote storage service provider 102, aformat of the data 500 as it will be stored at the remote storageservice provider 102).

FIG. 16 is a flow diagram of a process for storing encrypted shards ontodifferent shippable storage devices 100, according to some embodiments.One or more portions of the illustrated process may be performed via thedata transfer tool 522.

At block 1602, the data transfer tool 522 identifies data 500 on thecustomer network 104 to be imported into the storage service provider102. In some embodiments, the data transfer tool 522 may receive anindication of the data 500 to be imported, such as by a user interfaceassociated with the data transfer tool 522. In some instances, the datatransfer tool 522 determines the data 500 to be imported based oninformation in the job manifest 1300.

At block 1604, the data transfer tool 522 obtains the identified data500 for processing. For example, the data transfer tool 522 may copy theidentified data 500 into one or more portions of memory associated withthe data transfer tool 522.

At block 1606, the data transfer tool 522 generates shards (e.g.,fragments of the identified data 500) for the obtained data based on aredundancy encoding scheme. In some embodiments, the data transfer tool522 may generate two or more shards, wherein each of the two or moreshards, after being encrypted, are to be stored at two or morecorresponding shippable storage devices 100 that are attached to thecustomer network 104. In embodiments, the data transfer tool 522generates the two or more shards by applying a redundancy encodingtechnique (e.g., erasure coding) on the identified data 500. In someembodiments, the identified data may be broken into two or more shardsand encoded with portions of redundant data. When the two or more shardsare received and decrypted by the storage service provider 102, theidentified data 500 may be reconstructed from the two or more shards andimported into the storage service provider 102.

At block 1608, the data transfer tool 522 encrypts each of the generatedshards. In some embodiments, the encryption of each of the generatedshards is performed in the same or similar manner as the encryptiondescribed for FIG. 15. In some embodiments, each shard is treated as onefile before being encrypted. In some instances, each shard is split intotwo or more files before being encrypted.

At block 1610, the data transfer tool 522 determines a subset of theencrypted shards to import into the storage service provider 102, basedon a minimum number of shards required to reconstitute data stored amongthe shards. For example, if a minimum of six out of ten shards arerequired, then the data transfer tool 522 may determine at least six outof ten total shards to import into the storage service provider. Inembodiments, one or more shards remain stored by the customer (e.g., onone or more storage devices associated with the customer network), incase one or more shards are lost or intercepted during shipment.

At block 1612, the data transfer tool 522 assigns each encrypted shardof the subset of encrypted shards to a corresponding shippable storagedevice 100. At block 1614, the data transfer tool 522 stores each of theassigned encrypted shards to the corresponding shippable storage device100. The data transfer tool 522 may transfer two or more of theencrypted shards to two of more corresponding shippable storage devices100 in parallel. Transferring in parallel may reduce the amount of timerequired for the transfer. In some embodiments, if one or more of thecorresponding shippable storage device 100 are not yet attached to thecustomer network 104, then the data transfer tool 522 keeps thecorresponding encrypted shards stored in the virtual file system orother storage location at the customer network 104 until the one or moreof the corresponding shippable storage devices 100 are received andattached to the customer network 104.

FIG. 17 illustrates a process for encrypting chunks 1700 using chunkkeys 1702, according to some embodiments. One or more portions of theillustrated process may be performed via the data transfer tool 522. Thedata transfer tool 522 may identify customer data 500 to be encryptedand copied to the shippable storage device 100. The identified customerdata 500 may include at least one file 1704. In some embodiments, thedata transfer tool 522 copies the identified customer data 500 to avirtual file system before any processing or encrypting of the customerdata 500 is performed.

In some embodiments, the data transfer tool 522 divides a given file1704 into two or more different chunks 1700. For example, the datatransfer tool 522 may divide the file 1704 into n chunks 1700. In someembodiments, each chunk 1700 may be of equal size or approximately equalsize (e.g., 1 megabyte, 1 gigabyte, etc.). Each chunk 1700 may includeadditional information identifying the file 1704 that the chunk 1700belongs to. The additional information may also indicate what order thechunk 1700 is relative to other chunks (e.g., a sequence number), whichmay be used at a later point when assembling the chunks 1700 back intothe file 1704.

In various embodiments, the data transfer tool 522 generates a chunk key1702 for each chunk and each chunk 1700 is encrypted with thecorresponding chunk key 1702 to create an encrypted chunk 1706. Forexample, chunk 1700 a may be encrypted with chunk key 1702 a to generatethe encrypted chunk 1706 a. In some embodiments, each of the chunk keys1702 is different than the other chunk keys 1702. For example, the chunkkey 1702 a may be different than the chunk key 1702 b. In otherembodiments, each of the chunk keys 1702 may be the same key. Further,in some embodiments, at least two of the chunk keys 1702 are the same.For example, the chunk key 1702 a may be the same as the chunk key 1702b. Thus, in some embodiments, multiple chunks 1700 may be encrypted by aparticular chunk key 1702.

In some embodiments, the number of unique chunk keys 1702 that are usedto encrypt the chunks 1700 of a file 1704 depend on the size of the file1704. For example, if the file 1704 is less than a threshold size (e.g.,one gigabyte), then each chunk 1700 may be encrypted by a differentchunk key 1702. If the file 1704 is equal to or greater than a thresholdsize (e.g., one gigabyte), then a particular number of chunks 1700 mayby encrypted by the each unique chunk key 1702. For example, for a file1704 that is greater than one gigabyte in size, each unique chunk key1702 may be used to encrypt two chunks 1700 (unless there is an oddnumber of chunks, in which one chunk may be encrypted by a particularkey).

Furthermore, in some embodiments, any number of chunks may be assignedto each unique chunk key. In some instances, if the file 1704 is equalto or greater than a threshold size (e.g., one gigabyte), then oneunique chunk key 1702 may be used to encrypt all chunks 1700 within aparticular portion of the file 1704. For example, a first chunk key 1206may be used to encrypt all chunks within the first gigabyte of the file1704, a second chunk key 1206 may be used to encrypt all chunks within asecond gigabyte of the file 1704, and a third chunk key 1206 may be usedto encrypt all chunks within a third gigabyte of the file 1704 (or theremainder of the file 1704, if less than one gigabyte remains).

In some embodiments, after each chunk key 1702 encrypts a correspondingchunk 1700, the chunk key 1702 is encrypted by another key and thenstored or otherwise associated with the encrypted chunk 1700. Forexample, the encrypted chunk key may be stored adjacent to the encryptedchunk 1700 or within a particular offset of the encrypted chunk 1700. Insome embodiments, an ID associated with a chunk key 1702 and theencrypted chunk 1700 may be used to associate the chunk key 1702 withthe encrypted chunk 1700, which may be useful for decrypting theencrypted chunks 1208 at a later point in time. The identifier for thefile key (e.g., file key ID) that decrypts the chunk key may also bestored in association with the encrypted chunk and/or the encryptedchunk key. Therefore, in embodiments, the provision and ingestionservice 520 may determine for each encrypted chunk, the correspondingchunk key for decrypting the chunk as well as the corresponding file keyfor decrypting the chunk key.

Furthermore, in embodiments, the encrypted chunks 1706 are organizedinto subsets of chunks, where each subset corresponds to a particularfile 1704. In some embodiments, a file record is generated for each file1704. The file record may include data and metadata associated with thefile 1704. For example, the metadata may include a file key ID and abucket key ID that corresponds to a particular bucket on the storageservice provider 102 that the file 1704 is assigned to be imported to.Therefore, during decryption at the storage service provider, theprovision and ingestion service 520 may determine for each file record,the corresponding chunks that can be assembled to form the file (e.g.,by finding chunks with file key ID's that match the file key ID of thefile record).

In some embodiments, before any decryption takes place, each encryptedchunk is stored in association with (e.g., adjacent to, within aparticular offset, or otherwise associated with) an encrypted chunk keythat is used (in decrypted form) to decrypt the encrypted chunk. Theidentifier for the file key that decrypts the chunk key may also bestored in association with the encrypted chunk and/or the encryptedchunk key. Therefore, in embodiments, the provision and ingestionservice 520 may determine for each encrypted chunk, the correspondingchunk key for decrypting the chunk as well as the corresponding file keyfor decrypting the chunk key

FIG. 18 illustrates a process for encrypting chunk keys 1702 using afile key 1800, according to some embodiments. One or more portions ofthe illustrated process may be performed via the data transfer tool 522.In embodiments, the data transfer tool 522 generates a file key 1800 foreach file 1704. Thus, the file key 1800 may be associated with the file1704. The file key 1800 may be used to encrypt the chunk keys 1702 athrough 1702 n to generate the encrypted chunk keys 1802 a through 1802n. Further, in some embodiments, if the file 1704 is smaller than athreshold size (e.g., one gigabyte), then the file key 1800 may be usedto encrypt the encrypted chunks 1706.

FIG. 19 illustrates a process for encrypting file keys 1800 using abucket key 1900, according to some embodiments. The file keys 1800 athrough 1800 n may each be associated with corresponding files of thecustomer data 500 within the virtual file system. The bucket key 1900may be obtained from the manifest 1300 and used to encrypt the file keys1800 a through 1800 n to generate the encrypted file keys 1902 a through1902 n.

In some embodiments, the files associated with file keys 1800 a through1800 n are assigned to a particular storage location of the storagedevice 502 of the storage service provider 102. For example, the filesassociated with file keys 1800 a through 1800 n may be assigned to aparticular bucket of the storage device 502. Further, the bucket key1900 may also be assigned to the particular bucket. Since the file keys1800 and the bucket key 1900 are both assigned to the same bucket, thebucket key 1900 may be used to encrypt the file keys 1800 a through 1800n. Similarly, other files of customer data 500 that are to be stored ina different bucket of the storage device 502 may be encrypted with otherbucket keys that correspond to the different bucket.

In some embodiments, multiple bucket keys are assigned to each bucket.Therefore, some files destined for storage at a particular bucket may beencrypted with a first bucket key associated with the particular bucketwhile other files destined for storage at the same bucket may beencrypted with a second bucket key associated with the particularbucket. In various embodiments, any number of bucket keys may beassigned to each bucket (e.g., 2, 10, etc.).

FIG. 20 is a logical block diagram of relationship between encryptionkeys associated with a data import job, according to some embodiments.In some embodiments, one or more location keys are obtained from themanifest 1300, wherein each location key corresponds to a location forstoring data on the storage device 502 of the storage service provider102. For example, bucket keys 1900 a-n may be obtained from the manifest1300. The bucket keys 1900 a-n may also be stored in a secure locationof the data storage provider 102, such as key data 506.

In some embodiments, each bucket key 1900 is used to encrypt one or morefile keys 1800 a-n. In some instances, the one or more file keys 1800a-n are generated at the customer network 104, such as by the datatransfer tool 522. In some embodiments, at least some of the one or morefile keys 1800 a-n are obtained from the storage service provider 102,such as via the manifest 1300.

In some instances, each file key 1800 is used to encrypt one or morechunk keys 1700 a-n. As discussed above, in some embodiments, one ormore additional intermediate levels of file keys 1800 may be used to addextra levels of encryption in between the bucket keys 1900 and the chunkkeys 1702. In some instances, the one or more chunk keys 1700 a-n aregenerated at the customer network 104, such as by the data transfer tool522. In some embodiments, at least some of the one or more chunk keys1700 a-n are obtained from the storage service provider 102, such as viathe manifest 1300.

FIG. 21 is a logical block diagram of a shippable storage device 100ready for shipping to a storage service provider 102, according to someembodiments. After the data transfer tool 522 completes encryption ofthe customer data 500 and encryption keys, the data transfer tool 522may transfer the encrypted chunks 1706, the encrypted chunk keys 2102,and the encrypted file keys 2104 from the virtual file system to thepersistent data storage 902 of the shippable storage device 100.

In various embodiments, the persistent data storage 902 may also includeadditional data such as the address information 904 and the securityinformation 906, as described above for FIG. 9. The display 108 may alsodisplay the storage server provider address 2200, obtained from theaddress information 904. For example, in response to receiving theencrypted chunks 1706, the encrypted chunk keys 2102, and the encryptedfile keys 2104, the shippable storage device 100 may display the addressof the storage service provider 102 on the display 108. In someinstances, the data transfer tool 522 may send the storage serverprovider address 2200 to the shippable storage device 100 for thedisplay 108.

Since there is no key stored on the shippable storage device 100 fordecrypting the encrypted chunks 1902 or the encrypted keys, a thirdparty that somehow obtains the device during transit and accesses thepersistent data storage 902 may be unable to decrypt the encryptedchunks 1706, the encrypted chunk keys 2102, and the encrypted file keys2104. In some embodiments, some or all of the other data within thepersistent data storage 902 is also encrypted. For example, the datatransfer tool 522 may copy/remove any of the other data including theaddress information 706, the security information 708, or otherinformation from the shippable storage device 100, encrypt the data(e.g., with a bucket key 1900 or another key), then store the encrypteddata on the storage device 100. In some instances, at least someportions of the address information and/or security information mayremain in clear text form. For example, the address information may bestored in clear text form in order for the destination address to bedisplayed on the display 108.

FIG. 22 is a flow diagram of a process for receiving a shippable storagedevice 100 at a storage service provider 102 and ingesting secure datafrom the shippable storage device 100, according to some embodiments.One or more portions of the illustrated process may be performed by thestorage service provider 102, such as by the provision and ingestionservice 520 and/or other services.

Furthermore, in some embodiments, the storage service provider 102retrieves the encrypted data and keys from two or more shippable storagedevices 100. For example, the processes of FIGS. 22, 23, and/or 24 maybe performed for two or more shippable storage devices, in serial orconcurrently in parallel. When performing any of the processes inparallel for two or more shippable storage devices 100, the amount oftime required to import data into the storage service provider 102 maybe reduced, compared to importing the same data serially or by usingjust one shippable storage device 100.

In some embodiments, two or more shards may be shipped to the storageservice provider 102 on two or more corresponding shippable storagedevices 100, as described for FIG. 16. After the provision and ingestionservice 520 receives and decrypts all of the shards from each of the twoor more corresponding shippable storage devices 100, the provision andingestion service 520 may then reconstitute the data 500 to be importedinto the storage service provider 102.

At block 2202, after receiving and inspecting the shippable storagedevice 100, a user at the storage service provider connects theshippable storage device 100 to a network of the storage serviceprovider 102. In some embodiments, the shippable storage device 100 isinspected visually to determine that the shippable storage device 100was not tampered with or damaged. In embodiments, the shippable storagedevice 100 may also be scanned or otherwise inspected using one or moretechnologies that may not require physical contact with the device.

After the shippable storage device 100 passes a visual and/or physicalinspection, the device may be connected to the network. In someinstances, a guard service of the storage service provider 102communicates with the device after connection. The guard service may berunning on a network that is logically and/or physically separate fromone or more other networks of the storage service provider 102 in orderto prevent a possible network attack or software attack, such as from asoftware virus. In some embodiments, the storage service provider 102may identify the shippable storage device 100 based on obtaining adevice ID or other information from the shippable storage device 100.

At block 2204, the storage service provider 102 validates the shippablestorage device 100. In some embodiments, the storage service provider102 determines whether the configuration has changed since the shippablestorage device 100 was configured, provisioned, and shipped from thestorage service provider 102. In some instances, the storage serviceprovider 102 determines whether the configuration has changed since theshippable storage device 100 was configured and shipped from thecustomer to the storage service provider 102. The configuration of theshippable storage device 100 may include one or more of a physicalconfiguration of one or more components, a firmware configuration, anoperating system configuration, and other software configuration.

In some embodiments, the storage service provider 102 determines whetherthe configuration of the shippable storage device 100 has changed atleast in part by communicating with the TPM 360 and analyzing dataobtained from the TPM 360. The data obtained from the TPM 360 mayindicate, based on the analysis, that the shippable storage device 100was tampered with and/or that a third party may have accessed data onthe shippable storage device 100.

If the storage service provider 102 determines that the configuration ofthe shippable storage device 100 has changed, then the storage serviceprovider 102 may wipe the shippable storage device 100. In someembodiments, the storage service provider 102 indicates that processingof the shippable storage device 100 is to be put on hold and awaitinspection.

At block 2206, the storage service provider 102 determines the dataimport job associated with the shippable storage device 100. In someembodiments, the storage service provider 102 may determine an importjob ID (or other import job identifier) for the shippable storage device100 based on obtaining a device ID (e.g. a mac address specific to theshippable storage device 100) from the shippable storage device 100. Thestorage service provider 102 may have database or other storage recordsthat associate the particular import job (e.g., via a job ID) with themac address. In some embodiments, the storage service provider 102determines the data import job associated with the shippable storagedevice 100 based on the security information 906.

At block 2208, in response to determining the data import job associatedwith the shippable storage device 100, the storage service provider 102retrieves security information and encryption keys (e.g., bucket keysassigned to the customer) associated with the identified data import jobfrom one or more storage location of the storage service provider 102,such as key data 506 and metadata 508. In some instances, additional thestorage service provider 102 may retrieve additional data associatedwith the data import job. In some embodiments, the storage serviceprovider 102 may have database or other storage records that associatethe import job ID and/or the device ID with the security information 708for authenticating the shippable storage device 100 as well asencryption keys that were used for encrypting data on the shippablestorage device 100, such as the bucket keys 1900. In some embodiments,the security information retrieved from the storage service provider 102includes a root certificate or key that can be used to authenticate theshippable storage device 100.

In some embodiments, the storage service provider 102 may retrieve atleast some of the file keys 1502 and/or at least some of the chunk keys1206 that were generated by the data transfer tool 522 from a storagelocation of the storage service provider 102. For example, in someembodiments, the data transfer tool 522 may transmit at least some ofthe generated file keys 1502 and/or the generated chunk keys 1206 to thestorage service provider 102 instead of encrypting them and storing themon the shippable storage device.

At block 2210, the storage service provider 102 uses the retrievedsecurity information to authenticate the shippable storage device 100.For example, the storage service provider 102 may authenticate theshippable storage device 100 based on a root certificate or key found inthe security information and a corresponding root certificate or keystored on the shippable storage device 100. In embodiments, theshippable storage device 100 may perform a similar authentication, sothat the storage service provider 102 and the shippable storage device100 mutually authenticate.

In embodiments, the security information 906 on the shippable storagedevice 100 includes a certificate which includes one part of anasymmetric key, and the security information of the 1302 manifest 1300includes the other part of the asymmetric key. Thus, the authenticationmay be based on a public-private key pair. In some embodiments, thesecurity information 906 on the shippable storage device 100 includesboth a public and a private key and the security information of the 1302manifest 1300 includes a corresponding private and a correspondingpublic key that allows for the mutual authentication. In embodiments,the certificate used for authentication is specific to the particulardata import job that the shippable storage device 100 is being used for.In embodiments, if the data transfer tool 522 is unable to authenticatethe shippable storage device 100 based on the certificate, then the datatransfer tool 522 is unable to transfer any encrypted or unencrypteddata to the shippable storage device.

In some embodiments, in response to authenticating the shippable storagedevice 100 and verifying that the shippable storage device 100 does nothave any malicious code, the storage service provider 102 may switchfrom using the guard service to using the provision and ingestionservice 520 to communicate with the shippable storage device 100.

At block 2212, the storage service provider 102 retrieves the encrypteddata and keys from the shippable storage device 100. For example,storage service provider 102 may retrieve the encrypted chunks 1706, theencrypted chunk keys 2102, and the encrypted file keys 2104 from theshippable storage device 100. In some embodiments, the provision andingestion service 520 receives and transmits data to the shippablestorage device 100. The provision and ingestion service 520 may alsocommunicate with the job orchestrator 514 and the metadata service 512to obtain information for retrieving data and organizing data from theshippable storage device 100.

In some embodiments, the encrypted chunks 1706, the encrypted chunk keys2102, and the encrypted file keys 2104 are copied to a memory of thestorage service provider 102, such as volatile and/or non-volatilememory associated with the provision and ingestion service 520, beforebeing decrypted/imported. In various embodiments, the provision andingestion service 520 retrieves portions of the encrypted data and keysfrom the shippable storage device 100 at a time, where each portion isseparately decrypted and imported before retrieving the next portion.For example, the provision and ingestion service 520 may retrieve datain individual chunks, groups of chunks, keys, groups of keys, or anyother suitable portion and/or category.

At block 2214, the storage service provider 102 decrypts the encryptedkeys and data. For example, the provision and ingestion service 520 maydecrypt the encrypted chunks 1706, the encrypted chunk keys 2102, andthe encrypted file keys 2104. The decryption process is described inmore detail in FIG. 24.

At block 2216, the storage service provider 102 assembles the decryptedchunks into files. In some embodiments, the provision and ingestionservice 520 may use metadata from the decrypted chunks to assemble thechunks into files. For example, the metadata may indicate that a subsetof the chunks belong to a particular file. The metadata may alsoindicate a sequential order of the chunks for assembling back into theparticular file.

At block 2218, the storage service provider 102 the file integrity of afile. For example, the provision and ingestion service 520 may retrievea checksum for a particular file from the metadata service 512 (whichmay have been previously calculated and transmitted to the storageservice provider 102 by the data transfer tool 522), calculate achecksum for the particular file that was assembled, and then comparethe retrieved checksum to the calculated checksum. If the comparisonindicates a match, then the provision and ingestion service 520 mayverify the integrity of the file data.

At block 2220, in response to verifying the integrity of the file data,the storage service provider 102 stores the file in a bucket that thefile is assigned to. For example, the provision and ingestion service520 may store the file in a particular bucket within the storage device502 of the storage service provider 102.

At block 2222, the storage service provider 102 determines whether thereis another file to process for the data import job. If the storageservice provider 102 determines that there is another file to process,then the process returns to block 2218. If the storage service provider102 determines that there are no more files to process for the importjob, then the process proceeds to block 2224.

In some embodiments, two or more shards are received and decrypted bythe storage service provider 102. A shard may be decrypted as a singlefile or multiple files, depending on how the shards were encrypted. Thedata 500 to import may be reconstructed from the two or more shards andimported into the storage service provider 102. The two or more shardsmay be retrieved from one or multiple shippable storage devices 100.

At block 2224, the storage service provider 102 wipes the shippablestorage device 100. In some embodiments, the provision and ingestionservice 520 erases data and operating software from the shippablestorage device 100. The shippable storage device 100 may then bedisconnected from the network and moved to a storage area for wipeddevices that are ready to be provisioned. In some embodiments, theshippable storage device 100 remains connected to the network so that itcan be provisioned for another data import job for the same customer oranother customer.

FIG. 23 is a flow diagram of a process for decrypting and importing datafrom a shippable storage device 100 at a storage service provider 102,according to some embodiments. One or more portions of the illustratedprocess may be performed by one or more services of the storage serviceprovider 102, such as by the provision and ingestion service 520 and/orother services.

In some embodiments, the decryption of the keys and data may beperformed on subsets of the encrypted keys and/or encrypted data at atime, after transferring each respective subset of the encrypted keysand/or encrypted data to volatile and/or non-volatile memory of thestorage service provider 102, such as memory associated with theprovision and ingestion service 520. For example, as illustrated below,an encrypted file key for a particular file and encrypted chunks for theparticular file may be transferred from the shippable storage device 100to the data storage provider 102, the associated data may be decryptedand/or imported, then the provision and ingestion service 520 may repeatthe process for each additional file. In some embodiments, the provisionand ingestion service 520 may transfer all or a substantial portion ofthe encrypted data and keys from the shippable storage device 100 to thedata storage provider 102 and then perform the decryption and/orimporting of the data. In various embodiments, any other suitable subsetof keys and/or data may be transferred at a time from the shippablestorage device 100 to the data storage provider 102 for processing(e.g., decryption of keys and chunks, assembling chunks into a file,verifying file, storing file, etc.).

At block 2302, the storage service provider 102 retrieves the bucketkeys 1900 from a storage location of the storage service provider 102based on determining the data import job associated with the shippablestorage device 100. For example, the provision and ingestion service 520may obtain a mac address, import job ID, or other identifier from theshippable storage device 100 and retrieve the bucket keys 1900 from keydata 506 based on the mac address, job ID, or other identifier matchingone or more records stored by the storage service provider 102. In someembodiments, the provision and ingestion service 520 may also retrieveone or more other keys or data associated with the data import job.

At block 2304, the provision and ingestion service 520 determineswhether there is another bucket (e.g. a particular logical and/orphysical storage location for one or more files) that will be used tostore data imported from the shippable storage device 100. In someembodiments, this may include determining whether there is anotherbucket ID (or other location identifier) stored on the shippable storagedevice 100. If the provision and ingestion service 520 determines thereis another bucket, then the process continues to block 2306 to continueprocessing the file for the current bucket. If the provision andingestion service 520 determines there is not another bucket, then theprocess continues to block 2320, where the storage service provider 102may send a notification to one or more other services and/or devicesthat the data import for the data import job is complete.

At block 2306, the provision and ingestion service 520 determineswhether there is another file record (e.g. file data and metadataassociated with the file) associated with the bucket identified in block2304. In some embodiments, this includes determining if any more of theremaining file records (e.g., stored on the shippable storage device)include a bucket key ID that matches the bucket currently beingprocessed (determined at block 2304). If the provision and ingestionservice 520 determines there is another file record, then the processcontinues to block 2308. If the provision and ingestion service 520determines there is not another file record, then the process returns toblock 2304, where the storage service provider 102 determines whetherthere is data associated with another bucket that is to be imported forthe data import job. In some embodiments, a file record may include anencrypted file key that may be used in unencrypted form to decryptbucket keys associated with the file key, and a bucket key identifierthat identifies the bucket key associated with the bucket that the filewill be stored in.

At block 2308, the provision and ingestion service 520 obtains theencrypted file key of the file record (determined at block 2306) anddecrypts the encrypted file key using the bucket key (obtained at block2302) associated with the bucket (determined at block 2304). In someembodiments, the provision and ingestion service 520 determines that thefile record identifies a particular bucket key (and associated bucket)that was retrieved at block 2302. Therefore, the provision and ingestionservice 520 may use the identified bucket key to decrypt the file key ofthe file record.

At block 2310, the provision and ingestion service 520 obtains chunkkeys for the file and decrypts each of the chunk keys using the file keydecrypted at block 2308. At block 2312, the provision and ingestionservice 520 obtains the chunks for the file and decrypts each of thechunks with a corresponding chunk key obtained at block 2310. Forexample, each chunk key may be associated with a different chunk keythat is used to decrypt the particular chunk.

In some embodiments, before any decryption takes place, each encryptedchunk is stored in association with (e.g., adjacent to, within aparticular offset, or otherwise associated with) an encrypted chunk keythat is used (in decrypted form) to decrypt the encrypted chunk. Theidentifier for the file key that decrypts the chunk key may also bestored in association with the encrypted chunk and/or the encryptedchunk key. Therefore, in embodiments, the provision and ingestionservice 520 may determine for each encrypted chunk, the correspondingchunk key for decrypting the chunk as well as the corresponding file keyfor decrypting the chunk key.

At block 2312, the provision and ingestion service 520 obtains thechunks for the file and decrypts each of the chunks using thecorresponding chunk key (decrypted at block 2310). In some embodiments,the provision and ingestion service 520 determines which chunk key isused to decrypt a particular chunk based at least on metadata associatedwith the chunk and/or the chunk key. In some embodiments, the provisionand ingestion service 520 determines which chunk key is used to decrypta particular chunk based at least on a storage location of a chunk keywith respect to a storage location of the particular chunk.

At block 2314, the provision and ingestion service 520 assembles thechunks 1700 into the. In some embodiments, the provision and ingestionservice 520 may assemble the chunks into the file based at least onmetadata associated with each chunk (e.g., sequence number and/or orderwith respect to other chunks). In some instances, the provision andingestion service 520 may assemble the chunks into the file based atleast on a storage location of each chunk relative to one or more otherchunks. At block 2316, the provision and ingestion service 520 verifiesthe file using checksums, as described above. At block 2318, theprovision and ingestion service 520 stores the file into the bucket atthe storage service provider 102. The process then returns to block2306, where the provision and ingestion service 520 determines whetherthere is another file record for the bucket.

FIG. 24 is a flow diagram of a process for wiping a shippable storagedevice 100 at a storage service provider 102, according to someembodiments. One or more portions of the illustrated process may beperformed by one or more services of the storage service provider, suchas the provision and ingestion service 520 and/or other services.

At block 2402, the storage service provider 102 erases data from theshippable storage device 100. For example, the storage service provider102 may erase data from the persistent data storage 902.

At block 2404, the storage service provider 102 erases operatingsoftware from the shippable storage device 100. For example the storageservice provider 102 may the erase operating system and servers from thepersistent data storage 902.

At block 2406, the storage service provider 102 de-associates theshippable storage device 100 from the customer. For example, the storageservice provider 102 may remove one or more records associating thecustomer or customer network 104 with the shippable storage device 100.At block 2408, the storage service provider 102 moves the shippablestorage device 100 to a storage area for wiped shippable storage devices100. In some embodiments, the shippable storage device 100 may remainattached to the network and await provision for a new data import job.

Any of various computer systems may be configured to implement theprocesses associated (e.g., provisioning or ingestion by the serviceprovider or execution of the downloaded application on a customerserver) with a shippable storage device. For example, FIG. 25 is a blockdiagram illustrating one embodiment of a computer system suitable forimplementing some of the systems and methods described herein. Invarious embodiments, the storage service provider 102, or customercomputers at the customer network 104 (e.g., customer device 504 orcomputing device 1102) may each include a computer system 2500 such asthat illustrated in FIG. 25.

In the illustrated embodiment, computer system 2500 includes one or moreprocessors 2510 coupled to a system memory 2520 via an input/output(I/O) interface 2530. Computer system 2500 further includes a networkinterface 2540 coupled to I/O interface 2530. In some embodiments,computer system 2500 may be illustrative of servers implementingenterprise logic or downloadable application, while in other embodimentsservers may include more, fewer, or different elements than computersystem 2500.

In various embodiments, computer system 2500 may be a uniprocessorsystem including one processor 2510, or a multiprocessor systemincluding several processors 2510 (e.g., two, four, eight, or anothersuitable number). Processors 2510 may be any suitable processors capableof executing instructions. For example, in various embodiments,processors 2510 may be embedded processors implementing any of a varietyof instruction set architectures (ISAs), such as the x106, PowerPC,SPARC, or MIPS ISAs, or any other suitable ISA. In multiprocessorsystems, each of processors 2510 may commonly, but not necessarily,implement the same ISA.

System memory 2520 may be configured to store instructions and dataaccessible by processor 2510. In various embodiments, system memory 2520may be implemented using any suitable memory technology, such as staticrandom access memory (SRAM), synchronous dynamic RAM (SDRAM),non-volatile/Flash-type memory, or any other type of memory. In theillustrated embodiment, program instructions and data implementingdesired functions, such as those methods and techniques described abovefor the service provider or downloadable software are shown storedwithin system memory 2520 as program instructions 2525. In someembodiments, system memory 2520 may include data 2535 which may beconfigured as described herein.

In one embodiment, I/O interface 2530 may be configured to coordinateI/O traffic between processor 2510, system memory 2520 and anyperipheral devices in the system, including through network interface2540 or other peripheral interfaces. In some embodiments, I/O interface2530 may perform any necessary protocol, timing or other datatransformations to convert data signals from one component (e.g., systemmemory 2520) into a format suitable for use by another component (e.g.,processor 2510). In some embodiments, I/O interface 2530 may includesupport for devices attached through various types of peripheral buses,such as a variant of the Peripheral Component Interconnect (PCI) busstandard or the Universal Serial Bus (USB) standard, for example. Insome embodiments, the function of I/O interface 2530 may be split intotwo or more separate components, such as a north bridge and a southbridge, for example. Also, in some embodiments, some or all of thefunctionality of I/O interface 2530, such as an interface to systemmemory 2520, may be incorporated directly into processor 2510.

Network interface 2540 may be configured to allow data to be exchangedbetween computer system 2500 and other devices attached to a network,such as other computer systems, for example. In particular, networkinterface 2540 may be configured to allow communication between computersystem 2500 and/or various I/O devices 2550. I/O devices 2550 mayinclude scanning devices, display devices, input devices and/or othercommunication devices, as described herein. Network interface 2540 maycommonly support one or more wireless networking protocols (e.g.,Wi-Fi/IEEE 802.11, or another wireless networking standard). However, invarious embodiments, network interface 2540 may support communicationvia any suitable wired or wireless general data networks, such as othertypes of Ethernet networks, for example. Additionally, network interface2540 may support communication via telecommunications/telephony networkssuch as analog voice networks or digital fiber communications networks,via storage area networks such as Fibre Channel SANs, or via any othersuitable type of network and/or protocol.

In some embodiments, system memory 2520 may be one embodiment of acomputer-accessible medium configured to store program instructions anddata as described above. However, in other embodiments, programinstructions and/or data may be received, sent or stored upon differenttypes of computer-accessible media. Generally speaking, acomputer-accessible medium may include computer-readable storage mediaor memory media such as magnetic or optical media, e.g., disk orDVD/CD-ROM coupled to computer system 2500 via I/O interface 2530. Acomputer-readable storage medium may also include any volatile ornon-volatile media such as RAM (e.g. SDRAM, DDR SDRAM, RDRAM, SRAM,etc.), ROM, etc., that may be included in some embodiments of computersystem 2500 as system memory 2520 or another type of memory. Further, acomputer-accessible medium may include transmission media or signalssuch as electrical, electromagnetic, or digital signals, conveyed via acommunication medium such as a network and/or a wireless link, such asmay be implemented via network interface 2540.

In some embodiments, I/O devices 2550 may be relatively simple or “thin”client devices. For example, I/O devices 2550 may be configured as dumbterminals with display, data entry and communications capabilities, butotherwise little computational functionality. However, in someembodiments, I/O devices 2550 may be computer systems configuredsimilarly to computer system 2500, including one or more processors 2510and various other devices (though in some embodiments, a computer system2500 implementing an I/O device 2550 may have somewhat differentdevices, or different classes of devices).

In various embodiments, I/O devices 2550 (e.g., scanners or displaydevices and other communication devices) may include, but are notlimited to, one or more of: handheld devices, devices worn by orattached to a person, and devices integrated into or mounted on anymobile or fixed equipment, according to various embodiments. I/O devices2550 may further include, but are not limited to, one or more of:personal computer systems, desktop computers, rack-mounted computers,laptop or notebook computers, workstations, network computers, “dumb”terminals (i.e., computer terminals with little or no integratedprocessing ability), Personal Digital Assistants (PDAs), mobile phones,or other handheld devices, proprietary devices, printers, or any otherdevices suitable to communicate with the computer system 2500. Ingeneral, an I/O device 2550 (e.g., cursor control device, keyboard, ordisplay(s) may be any device that can communicate with elements ofcomputing system 2500.

The various methods as illustrated in the figures and described hereinrepresent illustrative embodiments of methods. The methods may beimplemented manually, in software, in hardware, or in a combinationthereof. The order of any method may be changed, and various elementsmay be added, reordered, combined, omitted, modified, etc. For example,in one embodiment, the methods may be implemented by a computer systemthat includes a processor executing program instructions stored on acomputer-readable storage medium coupled to the processor. The programinstructions may be configured to implement the functionality describedherein (e.g., the functionality of the data transfer tool, variousservices, databases, devices and/or other communication devices, etc.).

Various modifications and changes may be made as would be obvious to aperson skilled in the art having the benefit of this disclosure. It isintended to embrace all such modifications and changes and, accordingly,the above description to be regarded in an illustrative rather than arestrictive sense.

Various embodiments may further include receiving, sending or storinginstructions and/or data implemented in accordance with the foregoingdescription upon a computer-accessible medium. Generally speaking, acomputer-accessible medium may include storage media or memory mediasuch as magnetic or optical media, e.g., disk or DVD/CD-ROM, volatile ornon-volatile media such as RAM (e.g. SDRAM, DDR, RDRAM, SRAM, etc.),ROM, etc., as well as transmission media or signals such as electrical,electromagnetic, or digital signals, conveyed via a communication mediumsuch as network and/or a wireless link.

What is claimed is:
 1. A system, comprising: one or more computingdevices connected to a network of a storage service provider; at leastone shippable storage device received from a client of the storageservice provider, wherein the at least one shippable storage device isattached to the network of the storage service provider; and a dataingestion service implemented on at least one of the one or morecomputing devices, wherein the data ingestion service is configured to:determine information for a data import job associated with theshippable storage device; obtain, based on the information for the dataimport job, security information and one or more stored keys stored bythe storage service provider; authenticate the shippable storage devicebased on the security information; obtain encrypted keys and encrypteddata from the shippable storage device; decrypt one or more of theencrypted keys using the one or more keys stored by the storage serviceprovider to generate decrypted keys; decrypt the encrypted data based onusage of the decrypted keys to generate decrypted data; and store thedecrypted data at one or more locations at the storage service providerindicated by the information for the data import job.
 2. The system asrecited in claim 1, wherein the encrypted keys comprise one or moreencrypted chunk keys and the decrypted keys comprise one or more filekeys, and wherein to decrypt the encrypted data based on usage of thedecrypted keys, the data ingestion service is further configured to:decrypt the one or more encrypted chunk keys using the one or more filekeys to generate one or more chunk keys; and decrypt one or moreencrypted chunks of data using the one or more chunk keys to generatethe decrypted data.
 3. The system as recited in claim 1, wherein thedata ingestion service is further configured to: combine two or more ofthe decrypted chunks of data to form a file.
 4. The system as recited inclaim 1, wherein the data ingestion service is further configured to:receive configuration information from a trusted platform module of theshippable storage device; and determine, based on the configurationinformation, that a configuration of the shippable storage device hasnot changed since the shippable storage device was provisioned and sentto the client.
 5. The system as recited in claim 1, wherein the dataingestion service is further configured to: erase the shippable storagedevice to remove operating software and data associated with the clientfrom the shippable storage device.
 6. The system as recited in claim 1,wherein the decrypted data comprises a shard, and wherein the dataingestion service is configured to: reconstruct data based on the shardand at least one other shard; and store the reconstructed data at theone or more locations at the storage service provider.
 7. A method,comprising: performing, by a data ingestion service implemented on oneor more computing devices of a storage service provider: determining adata import job associated with a shippable storage device received froma client, wherein the shippable storage device is attached to a networkof the storage service provider; obtaining, based on the data importjob, one or more stored keys stored by the storage service provider;obtaining encrypted keys and encrypted data from the shippable storagedevice; decrypting one or more of the encrypted keys using the one ormore stored keys to generate one or more decrypted keys; decrypting theencrypted data based at least on usage of the one or more decrypted keysto generate decrypted data; and storing the decrypted data at one ormore locations at the storage service provider.
 8. The method as recitedin claim 7, wherein the encrypted keys comprise one or more encryptedchunk keys and the one or more decrypted keys comprise one or more filekeys, and further comprising: decrypting the one or more encrypted chunkkeys using the one or more file keys to generate one or more chunk keys;and decrypting one or more encrypted chunks of data using the one ormore chunk keys to generate one or more decrypted chunks of data.
 9. Themethod as recited in claim 8, further comprising: combining two or moreof the decrypted chunks of data to form a file.
 10. The method asrecited in claim 9, further comprising: calculating a checksum for thefile; comparing the calculated checksum to a checksum stored by thestorage service provider; and verifying the integrity of the file inresponse to determining that the calculated checksum matches the storedchecksum.
 11. The method as recited in claim 7, wherein the decrypteddata comprises a shard, and further comprising: reconstructing databased on the shard and at least one other shard; and storing thereconstructed data at the one or more locations at the storage serviceprovider.
 12. The method as recited in claim 7, further comprising:receiving configuration information from a trusted platform module ofthe shippable storage device; and determining, based on theconfiguration information, that a configuration of the shippable storagedevice has not changed since the shippable storage device wasprovisioned and sent to the client.
 13. The method as recited in claim7, further comprising: erasing the shippable storage device to removedata associated with the client from the shippable storage device.
 14. Anon-transitory computer-accessible storage medium storing programinstructions that, when executed on one or more processors of a dataingestion service cause the one or more processors to implement a dataingestion service to perform: determining a data import job associatedwith a shippable storage device received from a client, wherein theshippable storage device is attached to a network of the storage serviceprovider; obtaining, based on the data import job, one or more storedkeys stored by the storage service provider; obtaining encrypted keysand encrypted data from the shippable storage device; decrypting one ormore of the encrypted keys using the one or more stored keys to generateone or more decrypted keys; decrypting the encrypted data based at leaston usage of the one or more decrypted keys to generate decrypted data;and storing the decrypted data at one or more locations at the storageservice provider.
 15. The computer-accessible storage medium as recitedin claim 14, wherein the encrypted keys comprise one or more encryptedchunk keys and the one or more decrypted keys comprise one or more filekeys, and wherein the program instructions when executed cause the oneor more processors to perform: decrypting the one or more encryptedchunk keys using the one or more file keys to generate one or more chunkkeys; and decrypting one or more encrypted chunks of data using the oneor more chunk keys to generate one or more decrypted chunks of data. 16.The computer-accessible storage medium as recited in claim 15, whereinthe program instructions when executed cause the one or more processorsto perform: combining two or more of the decrypted chunks of data toform a file.
 17. The computer-accessible storage medium as recited inclaim 16, wherein the program instructions when executed cause the oneor more processors to perform: calculating a checksum for the file;comparing the calculated checksum to a checksum stored by the storageservice provider; and verifying the integrity of the file in response todetermining that the calculated checksum matches the stored checksum.18. The computer-accessible storage medium as recited in claim 14,wherein the decrypted data comprises a shard, and wherein the programinstructions when executed cause the one or more processors to perform:reconstructing data based on the shard and at least one other shard; andstoring the reconstructed data at the one or more locations at thestorage service provider.
 19. The computer-accessible storage medium asrecited in claim 14, wherein the program instructions when executedcause the one or more processors to perform: receiving configurationinformation from a trusted platform module of the shippable storagedevice; and determining, based on the configuration information, that aconfiguration of the shippable storage device has not changed since theshippable storage device was provisioned and sent to the client.
 20. Thecomputer-accessible storage medium as recited in claim 14, the programinstructions when executed cause the one or more processors to perform:obtaining, based on the data import job, a root certificate or anotherkey stored by the storage service provider; and authenticating theshippable storage device based at least on the root certificate or theother key.